2024-05-21T12:04:01.240 INFO:root:teuthology version: 0.0.1.dev288+g89d7af0 2024-05-21T12:04:01.240 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi --archive /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477 --owner scheduled_vallariag@teuthology --description rbd:nvmeof/thrash/{base/install centos_latest clusters/3-gateways-1-initiator conf/{disable-pool-app mon} gateway-initiator-setup/3-subsys-60-namespace thrashers/nvmeof_thrash workloads/fio} -- /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/orig.config.yaml 2024-05-21T12:04:01.270 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T12:04:01.357 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477 branch: main description: rbd:nvmeof/thrash/{base/install centos_latest clusters/3-gateways-1-initiator conf/{disable-pool-app mon} gateway-initiator-setup/3-subsys-60-namespace thrashers/nvmeof_thrash workloads/fio} email: null first_in_suite: false job_id: '7718477' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: mon warn on pool no app: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_FAILED_DAEMON - is in error state - failed cephadm daemon sha1: b0c764b6e22fbb51f59e469d9dd99895c152f73e ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: b0c764b6e22fbb51f59e469d9dd99895c152f73e selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: wip-nvmeof-test-v4-debug6 sha1: 68fa3266016634745d991c9967c479c757966afe owner: scheduled_vallariag@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - mon.a - mgr.x - osd.0 - osd.1 - client.0 - ceph.nvmeof.nvmeof.a - - host.b - mon.b - osd.2 - osd.3 - osd.4 - client.1 - ceph.nvmeof.nvmeof.b - - host.c - mon.c - osd.5 - osd.6 - osd.7 - client.2 - ceph.nvmeof.nvmeof.c - - client.3 seed: 9554 sha1: b0c764b6e22fbb51f59e469d9dd99895c152f73e sleep_before_teardown: 0 suite: rbd:nvmeof suite_branch: wip-nvmeof-test-v4-debug6 suite_path: /home/teuthworker/src/github.com_VallariAg_ceph_68fa3266016634745d991c9967c479c757966afe/qa suite_relpath: qa suite_repo: https://github.com/VallariAg/ceph suite_sha1: 68fa3266016634745d991c9967c479c757966afe targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBt8HBl9rkfobRjw31sKCwU+/vqvESfGZ8lJWg6H3UyeXCzr8guvHZGniTqhbUAmLrELkvU0HBV/0sFf0gF7dpc= smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBbIyv3Z7sAjVomk9hSONJTVtTDhp652j6TTXi1TpohEYm3zdRxDaKlEVtRjB5Jd/O4n+2bRn6HUFX3LAbYKNk= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBELi6y5o/PxH8xIWbUxJI1zLfmW/uJhOVH+FbREEw25tE9YkrlewXM1AqkQAd4zE5oo/WW2L073KpQU4YvzK4DU= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHxhEQ5yckel3PR5gbAf3jgL6s11BGyrdDS5xX2Mr6rrl8CFQoufG7gWvV2nPi9t2Yg6gL3dmi/p96IV5bhdh38= tasks: - install: extra_packages: - nvme-cli - cephadm: watchdog_setup: null - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch host ls - ceph orch device ls - ceph osd lspools - nvmeof: client: client.0 gateway_config: cli_image: quay.io/ceph/nvmeof-cli:1.2 namespaces_count: 20 subsystems_count: 3 gw_image: quay.io/ceph/nvmeof:1.2 rbd: image_name_prefix: myimage pool_name: mypool - cephadm.wait_for_service: service: nvmeof.mypool - workunit: clients: client.3: - rbd/nvmeof_setup_subsystem.sh - rbd/nvmeof_basic_tests.sh env: RBD_IMAGE_PREFIX: myimage RBD_POOL: mypool no_coverage_and_limits: true - nvmeof.thrash: checker_host: client.3 - workunit: clients: client.3: - rbd/nvmeof_fio_test.sh --rbd_iostat env: IOSTAT_INTERVAL: '10' RBD_POOL: mypool RUNTIME: '600' no_coverage_and_limits: true timeout: 30m teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: fix-install-task teuthology_sha1: 89d7af0c81df042065cab9effa470ce90a88fbde timestamp: 2024-05-21_11:53:37 tube: smithi use_shaman: true user: vallariag verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-21T12:04:01.357 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_VallariAg_ceph_68fa3266016634745d991c9967c479c757966afe/qa; will attempt to use it 2024-05-21T12:04:01.357 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_VallariAg_ceph_68fa3266016634745d991c9967c479c757966afe/qa/tasks 2024-05-21T12:04:01.358 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-21T12:04:01.359 INFO:teuthology.task.internal:Checking packages... 2024-05-21T12:04:01.380 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b0c764b6e22fbb51f59e469d9dd99895c152f73e' 2024-05-21T12:04:01.380 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-21T12:04:01.380 INFO:teuthology.packaging:ref: None 2024-05-21T12:04:01.380 INFO:teuthology.packaging:tag: None 2024-05-21T12:04:01.380 INFO:teuthology.packaging:branch: main 2024-05-21T12:04:01.380 INFO:teuthology.packaging:sha1: b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:04:01.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-05-21T12:04:01.646 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3823.gacdd6bc0 2024-05-21T12:04:01.648 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-21T12:04:01.655 INFO:teuthology.task.internal:no buildpackages task found 2024-05-21T12:04:01.655 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-21T12:04:01.661 INFO:teuthology.task.internal:Saving configuration 2024-05-21T12:04:01.673 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-21T12:04:01.680 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-21T12:04:01.707 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 11:58:37.507616', 'locked_by': 'scheduled_vallariag@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBt8HBl9rkfobRjw31sKCwU+/vqvESfGZ8lJWg6H3UyeXCzr8guvHZGniTqhbUAmLrELkvU0HBV/0sFf0gF7dpc='} 2024-05-21T12:04:01.728 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 11:58:37.503789', 'locked_by': 'scheduled_vallariag@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBbIyv3Z7sAjVomk9hSONJTVtTDhp652j6TTXi1TpohEYm3zdRxDaKlEVtRjB5Jd/O4n+2bRn6HUFX3LAbYKNk='} 2024-05-21T12:04:01.752 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 11:58:37.505350', 'locked_by': 'scheduled_vallariag@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBELi6y5o/PxH8xIWbUxJI1zLfmW/uJhOVH+FbREEw25tE9YkrlewXM1AqkQAd4zE5oo/WW2L073KpQU4YvzK4DU='} 2024-05-21T12:04:01.772 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 11:58:37.506464', 'locked_by': 'scheduled_vallariag@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHxhEQ5yckel3PR5gbAf3jgL6s11BGyrdDS5xX2Mr6rrl8CFQoufG7gWvV2nPi9t2Yg6gL3dmi/p96IV5bhdh38='} 2024-05-21T12:04:01.772 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-21T12:04:01.779 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['host.a', 'mon.a', 'mgr.x', 'osd.0', 'osd.1', 'client.0', 'ceph.nvmeof.nvmeof.a'] 2024-05-21T12:04:01.780 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['host.b', 'mon.b', 'osd.2', 'osd.3', 'osd.4', 'client.1', 'ceph.nvmeof.nvmeof.b'] 2024-05-21T12:04:01.780 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['host.c', 'mon.c', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'ceph.nvmeof.nvmeof.c'] 2024-05-21T12:04:01.780 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['client.3'] 2024-05-21T12:04:01.780 INFO:teuthology.run_tasks:Running task console_log... 2024-05-21T12:04:01.889 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f90faf7fac0>, signals=[15]) 2024-05-21T12:04:01.889 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-21T12:04:01.895 INFO:teuthology.task.internal:Opening connections... 2024-05-21T12:04:01.895 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-05-21T12:04:01.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:04:01.973 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2024-05-21T12:04:01.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:04:02.050 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-05-21T12:04:02.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:04:02.123 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-05-21T12:04:02.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:04:02.198 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-21T12:04:02.204 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-05-21T12:04:02.222 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-05-21T12:04:02.222 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-05-21T12:04:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T12:04:02.282 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-05-21T12:04:02.282 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T12:04:02.282 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T12:04:02.282 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T12:04:02.282 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-05-21T12:04:02.309 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2024-05-21T12:04:02.327 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2024-05-21T12:04:02.327 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="9" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="9" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el9" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:LOGO="fedora-logo-icon" 2024-05-21T12:04:02.384 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T12:04:02.385 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-05-21T12:04:02.385 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T12:04:02.385 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T12:04:02.385 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T12:04:02.385 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-05-21T12:04:02.418 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-05-21T12:04:02.435 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-05-21T12:04:02.435 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-05-21T12:04:02.492 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T12:04:02.493 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-05-21T12:04:02.493 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T12:04:02.493 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T12:04:02.493 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T12:04:02.493 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-05-21T12:04:02.526 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-05-21T12:04:02.544 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-05-21T12:04:02.544 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T12:04:02.601 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-05-21T12:04:02.602 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T12:04:02.602 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T12:04:02.602 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T12:04:02.602 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-05-21T12:04:02.643 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-21T12:04:02.652 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-21T12:04:02.658 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-21T12:04:02.658 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-05-21T12:04:02.660 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2024-05-21T12:04:02.663 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-05-21T12:04:02.665 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-05-21T12:04:02.681 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-21T12:04:02.687 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-21T12:04:02.687 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-05-21T12:04:02.718 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2024-05-21T12:04:02.720 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-05-21T12:04:02.722 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-05-21T12:04:02.753 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-21T12:04:02.849 INFO:teuthology.run_tasks:Running task kernel... 2024-05-21T12:04:02.864 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-21T12:04:02.864 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}, 'host.c': {'kdb': 1, 'sha1': 'distro'}, 'client.3': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-21T12:04:02.864 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T12:04:02.865 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T12:04:02.865 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T12:04:02.865 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T12:04:02.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:04:02.882 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-05-21T12:04:02.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:04:02.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:04:02.884 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2024-05-21T12:04:02.884 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-05-21T12:04:02.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:04:02.885 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-05-21T12:04:02.938 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-437.el9.x86_64 2024-05-21T12:04:02.938 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-437.el9.x86_64 2024-05-21T12:04:02.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-05-21T12:04:02.940 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-437.el9.x86_64 2024-05-21T12:04:02.940 INFO:teuthology.task.kernel:Running kernel on smithi137: 5.14.0-437.el9.x86_64 2024-05-21T12:04:02.941 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2024-05-21T12:04:02.942 INFO:teuthology.orchestra.run.smithi072.stdout:5.14.0-437.el9.x86_64 2024-05-21T12:04:02.942 INFO:teuthology.task.kernel:Running kernel on smithi072: 5.14.0-437.el9.x86_64 2024-05-21T12:04:02.942 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum install -y kernel 2024-05-21T12:04:02.943 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-437.el9.x86_64 2024-05-21T12:04:02.943 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-437.el9.x86_64 2024-05-21T12:04:02.943 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-05-21T12:04:05.294 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-21T12:04:05.861 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.1 MB 00:01 2024-05-21T12:04:06.591 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-21T12:04:07.315 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.1 MB 00:02 2024-05-21T12:04:08.686 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 19 MB 00:02 2024-05-21T12:04:09.111 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 19 MB 00:02 2024-05-21T12:04:10.642 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 19 MB 00:02 2024-05-21T12:04:14.703 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.2 MB 00:02 2024-05-21T12:04:16.058 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-05-21T12:04:16.069 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.2 MB 00:03 2024-05-21T12:04:16.146 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-21T12:04:17.529 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-21T12:04:17.709 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-21T12:04:18.838 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-21T12:04:20.185 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-05-21T12:04:22.466 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 21 MB 00:06 2024-05-21T12:04:23.971 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-21T12:04:25.330 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-21T12:04:25.350 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-21T12:04:25.844 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T12:04:25.844 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T12:04:25.935 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:04:25.944 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:04:25.944 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-05-21T12:04:25.944 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:04:25.944 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:04:25.944 INFO:teuthology.orchestra.run.smithi137.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T12:04:25.944 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:04:25.944 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T12:04:25.944 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T12:04:25.945 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T12:04:25.945 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:04:25.945 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:04:25.945 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:04:25.945 INFO:teuthology.orchestra.run.smithi137.stdout:Install 4 Packages 2024-05-21T12:04:25.945 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:04:25.946 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 78 M 2024-05-21T12:04:25.946 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 123 M 2024-05-21T12:04:25.946 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:04:27.114 INFO:teuthology.orchestra.run.smithi137.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 640 kB/s | 576 kB 00:00 2024-05-21T12:04:27.157 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T12:04:27.157 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T12:04:27.238 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:04:27.243 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:04:27.244 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-05-21T12:04:27.244 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:04:27.244 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 78 M 2024-05-21T12:04:27.244 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 123 M 2024-05-21T12:04:27.244 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:04:27.710 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 1.7 MB/s | 576 kB 00:00 2024-05-21T12:04:27.789 INFO:teuthology.orchestra.run.smithi137.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 9.9 MB/s | 16 MB 00:01 2024-05-21T12:04:27.821 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-05-21T12:04:28.548 INFO:teuthology.orchestra.run.smithi137.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 15 MB/s | 34 MB 00:02 2024-05-21T12:04:28.711 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-modules-core-5.14.0-447.el9.x86_6 28 MB/s | 28 MB 00:00 2024-05-21T12:04:29.406 INFO:teuthology.orchestra.run.smithi137.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 12 MB/s | 28 MB 00:02 2024-05-21T12:04:29.407 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:04:29.407 INFO:teuthology.orchestra.run.smithi137.stdout:Total 23 MB/s | 78 MB 00:03 2024-05-21T12:04:29.411 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 17 MB/s | 34 MB 00:02 2024-05-21T12:04:29.736 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T12:04:29.736 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T12:04:29.831 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:04:29.835 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-05-21T12:04:29.836 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:04:29.837 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 78 M 2024-05-21T12:04:29.837 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 123 M 2024-05-21T12:04:29.837 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:04:30.093 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:04:30.102 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-core-5.14.0-447.el9.x86_64.rpm 5.7 MB/s | 16 MB 00:02 2024-05-21T12:04:30.103 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:04:30.103 INFO:teuthology.orchestra.run.smithi092.stdout:Total 27 MB/s | 78 MB 00:02 2024-05-21T12:04:30.139 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 2.1 MB/s | 576 kB 00:00 2024-05-21T12:04:30.424 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:04:30.424 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:04:30.790 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:04:30.911 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:01 2024-05-21T12:04:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:04:31.143 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:04:31.338 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:04:31.338 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:04:32.047 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:04:32.048 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:04:32.471 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-05-21T12:04:33.114 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 4.8 MB/s | 16 MB 00:03 2024-05-21T12:04:33.707 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:04:34.118 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-21T12:04:34.326 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T12:04:34.407 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:04:34.411 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:04:34.889 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-core-5.14.0-447.el9.x86_6 5.9 MB/s | 28 MB 00:04 2024-05-21T12:04:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T12:04:35.144 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:04:35.551 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 6.0 MB/s | 34 MB 00:05 2024-05-21T12:04:35.551 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:04:35.551 INFO:teuthology.orchestra.run.smithi045.stdout:Total 14 MB/s | 78 MB 00:05 2024-05-21T12:04:36.265 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:04:36.660 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:04:36.660 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:04:37.059 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:04:37.184 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:04:37.623 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:04:37.624 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:04:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:04:37.946 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:04:39.184 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-21T12:04:40.216 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:04:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T12:04:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:04:41.067 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T12:04:41.067 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T12:04:41.189 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:04:41.214 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:04:41.214 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-05-21T12:04:41.214 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:04:41.214 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:04:41.214 INFO:teuthology.orchestra.run.smithi072.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T12:04:41.214 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:04:41.215 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T12:04:41.215 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T12:04:41.215 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T12:04:41.215 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:04:41.215 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:04:41.215 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:04:41.215 INFO:teuthology.orchestra.run.smithi072.stdout:Install 4 Packages 2024-05-21T12:04:41.215 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:04:41.216 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 78 M 2024-05-21T12:04:41.216 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 123 M 2024-05-21T12:04:41.216 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:04:41.621 INFO:teuthology.orchestra.run.smithi072.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 2.1 MB/s | 576 kB 00:00 2024-05-21T12:04:41.897 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:04:41.945 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:04:42.721 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:04:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:04:43.743 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:04:43.962 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:04:44.154 INFO:teuthology.orchestra.run.smithi072.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 5.5 MB/s | 16 MB 00:02 2024-05-21T12:04:45.980 INFO:teuthology.orchestra.run.smithi072.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 7.4 MB/s | 34 MB 00:04 2024-05-21T12:04:46.642 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:04:47.582 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:04:47.805 INFO:teuthology.orchestra.run.smithi072.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 4.5 MB/s | 28 MB 00:06 2024-05-21T12:04:47.805 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:04:47.805 INFO:teuthology.orchestra.run.smithi072.stdout:Total 12 MB/s | 78 MB 00:06 2024-05-21T12:04:48.490 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:04:48.833 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:04:48.833 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:04:49.029 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:04:49.081 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:04:49.765 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:04:49.765 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:04:52.201 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:04:52.896 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T12:04:53.002 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:04:54.180 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:04:55.640 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:04:55.851 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:05:00.597 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:05:00.642 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:05.395 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:15.781 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:15.820 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T12:05:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:05:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:05:16.868 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:16.906 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:17.008 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:17.009 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:05:17.009 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:05:17.009 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:17.009 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T12:05:17.009 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T12:05:17.009 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T12:05:17.009 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:05:17.009 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:05:17.212 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:17.213 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T12:05:17.213 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:05:17.213 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:05:17.399 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T12:05:17.466 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:17.466 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T12:05:17.466 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T12:05:17.466 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T12:05:17.466 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T12:05:17.467 INFO:teuthology.task.kernel:Installing distro kernel on client.3... 2024-05-21T12:05:17.467 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T12:05:17.467 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T12:05:17.517 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:17.517 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T12:05:17.517 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -qi grub2-tools 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:Name : grub2-tools 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:Epoch : 1 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:Version : 2.06 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:Release : 68.el9 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: x86_64 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:Group : Unspecified 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:Size : 8264254 2024-05-21T12:05:17.555 INFO:teuthology.orchestra.run.smithi137.stdout:License : GPLv3+ 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:Packager : builder@centos.org 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:Vendor : CentOS 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:Summary : Support tools for GRUB. 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:Description : 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T12:05:17.556 INFO:teuthology.orchestra.run.smithi137.stdout:hardware devices. 2024-05-21T12:05:17.557 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:05:17.557 INFO:teuthology.orchestra.run.smithi137.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T12:05:17.557 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T12:05:17.557 INFO:teuthology.task.kernel:Updating grub on smithi137 to boot 5.14.0-447.el9.x86_64 2024-05-21T12:05:17.558 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T12:05:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:18.009 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:05:18.009 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:05:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T12:05:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T12:05:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T12:05:18.010 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:05:18.010 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:05:18.170 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-05-21T12:05:18.359 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T12:05:18.433 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:18.433 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T12:05:18.433 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T12:05:18.434 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T12:05:18.434 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T12:05:18.434 INFO:teuthology.task.kernel:Installing distro kernel on host.c... 2024-05-21T12:05:18.434 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T12:05:18.434 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T12:05:18.484 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:18.484 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T12:05:18.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-05-21T12:05:18.514 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 68.el9 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8264254 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T12:05:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:05:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T12:05:18.517 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T12:05:18.517 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-447.el9.x86_64 2024-05-21T12:05:18.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T12:05:19.113 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-05-21T12:05:19.678 INFO:teuthology.orchestra.run.smithi137.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T12:05:19.704 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-05-21T12:05:19.707 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-05-21T12:05:19.722 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.yYHa5VOo77 2024-05-21T12:05:19.723 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yYHa5VOo77 2024-05-21T12:05:19.796 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.yYHa5VOo77 2024-05-21T12:05:19.938 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.yYHa5VOo77 is 6KB 2024-05-21T12:05:19.987 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.yYHa5VOo77 2024-05-21T12:05:20.001 DEBUG:teuthology.orchestra.run.smithi137:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T12:05:20.063 INFO:teuthology.orchestra.run.smithi137.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T12:05:20.063 INFO:teuthology.orchestra.run.smithi137.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T12:05:20.063 INFO:teuthology.orchestra.run.smithi137.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T12:05:20.063 INFO:teuthology.orchestra.run.smithi137.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T12:05:20.064 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T12:05:20.215 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-05-21T12:05:20.476 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T12:05:20.506 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-05-21T12:05:20.508 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-05-21T12:05:20.523 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.ZtV9Yu7D9I 2024-05-21T12:05:20.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZtV9Yu7D9I 2024-05-21T12:05:20.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.ZtV9Yu7D9I 2024-05-21T12:05:20.727 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.ZtV9Yu7D9I is 6KB 2024-05-21T12:05:20.746 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.ZtV9Yu7D9I 2024-05-21T12:05:20.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T12:05:20.827 INFO:teuthology.orchestra.run.smithi092.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T12:05:20.828 INFO:teuthology.orchestra.run.smithi092.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T12:05:20.828 INFO:teuthology.orchestra.run.smithi092.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T12:05:20.828 INFO:teuthology.orchestra.run.smithi092.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T12:05:20.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T12:05:20.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-05-21T12:05:25.350 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:25.390 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T12:05:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:05:25.743 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:05:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:05:26.534 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:05:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T12:05:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T12:05:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T12:05:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:05:26.535 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:05:26.897 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T12:05:26.974 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:26.974 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T12:05:26.974 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T12:05:26.974 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T12:05:26.974 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T12:05:26.975 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-05-21T12:05:26.975 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T12:05:26.975 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T12:05:27.026 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:27.027 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T12:05:27.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-05-21T12:05:27.061 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-05-21T12:05:27.061 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-05-21T12:05:27.061 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 68.el9 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8264254 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-05-21T12:05:27.062 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:05:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T12:05:27.064 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T12:05:27.064 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-447.el9.x86_64 2024-05-21T12:05:27.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T12:05:27.651 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-05-21T12:05:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T12:05:29.130 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-05-21T12:05:29.132 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-05-21T12:05:29.147 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.wxpKuZyWn2 2024-05-21T12:05:29.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wxpKuZyWn2 2024-05-21T12:05:29.213 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.wxpKuZyWn2 2024-05-21T12:05:29.351 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.wxpKuZyWn2 is 6KB 2024-05-21T12:05:29.400 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.wxpKuZyWn2 2024-05-21T12:05:29.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T12:05:29.478 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T12:05:29.478 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T12:05:29.478 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T12:05:29.478 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T12:05:29.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T12:05:29.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-05-21T12:05:36.969 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:37.006 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:37.412 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:37.412 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T12:05:37.412 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T12:05:37.413 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T12:05:38.451 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T12:05:38.451 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:05:38.451 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:05:38.451 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:38.452 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T12:05:38.452 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T12:05:38.452 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T12:05:38.452 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:05:38.452 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:05:38.902 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T12:05:38.993 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:38.993 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T12:05:38.994 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T12:05:38.994 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T12:05:38.994 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T12:05:38.994 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-05-21T12:05:38.994 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi072.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T12:05:38.994 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T12:05:39.045 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T12:05:39.045 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T12:05:39.045 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -qi grub2-tools 2024-05-21T12:05:39.080 INFO:teuthology.orchestra.run.smithi072.stdout:Name : grub2-tools 2024-05-21T12:05:39.080 INFO:teuthology.orchestra.run.smithi072.stdout:Epoch : 1 2024-05-21T12:05:39.080 INFO:teuthology.orchestra.run.smithi072.stdout:Version : 2.06 2024-05-21T12:05:39.080 INFO:teuthology.orchestra.run.smithi072.stdout:Release : 68.el9 2024-05-21T12:05:39.080 INFO:teuthology.orchestra.run.smithi072.stdout:Architecture: x86_64 2024-05-21T12:05:39.080 INFO:teuthology.orchestra.run.smithi072.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Group : Unspecified 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Size : 8264254 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:License : GPLv3+ 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Packager : builder@centos.org 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Vendor : CentOS 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Summary : Support tools for GRUB. 2024-05-21T12:05:39.081 INFO:teuthology.orchestra.run.smithi072.stdout:Description : 2024-05-21T12:05:39.082 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:05:39.082 INFO:teuthology.orchestra.run.smithi072.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T12:05:39.082 INFO:teuthology.orchestra.run.smithi072.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T12:05:39.082 INFO:teuthology.orchestra.run.smithi072.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T12:05:39.082 INFO:teuthology.orchestra.run.smithi072.stdout:hardware devices. 2024-05-21T12:05:39.082 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:05:39.082 INFO:teuthology.orchestra.run.smithi072.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T12:05:39.083 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T12:05:39.083 INFO:teuthology.task.kernel:Updating grub on smithi072 to boot 5.14.0-447.el9.x86_64 2024-05-21T12:05:39.083 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T12:05:39.766 INFO:teuthology.orchestra.run.smithi072.stderr:Generating grub configuration file ... 2024-05-21T12:05:41.336 INFO:teuthology.orchestra.run.smithi072.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T12:05:41.359 INFO:teuthology.orchestra.run.smithi072.stderr:done 2024-05-21T12:05:41.362 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2024-05-21T12:05:41.397 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.homGWDJQPv 2024-05-21T12:05:41.397 DEBUG:teuthology.orchestra.run.smithi072:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.homGWDJQPv 2024-05-21T12:05:41.461 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.homGWDJQPv 2024-05-21T12:05:41.600 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.homGWDJQPv is 6KB 2024-05-21T12:05:41.649 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.homGWDJQPv 2024-05-21T12:05:41.665 DEBUG:teuthology.orchestra.run.smithi072:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T12:05:41.724 INFO:teuthology.orchestra.run.smithi072.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T12:05:41.724 INFO:teuthology.orchestra.run.smithi072.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T12:05:41.724 INFO:teuthology.orchestra.run.smithi072.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T12:05:41.724 INFO:teuthology.orchestra.run.smithi072.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T12:05:41.725 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T12:05:41.863 DEBUG:teuthology.orchestra.run.smithi072:> sudo shutdown -r now 2024-05-21T12:05:50.226 INFO:teuthology.task.kernel:Checking client client.3 for new kernel version... 2024-05-21T12:05:50.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-05-21T12:05:50.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:05:50.994 INFO:teuthology.task.kernel:Checking client host.c for new kernel version... 2024-05-21T12:05:50.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-21T12:05:50.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:05:59.626 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-05-21T12:05:59.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-21T12:05:59.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:06:11.875 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-05-21T12:06:11.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-21T12:06:11.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:06:18.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-05-21T12:06:24.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-05-21T12:06:27.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-21T12:06:27.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:06:30.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-05-21T12:06:33.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-05-21T12:06:33.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:06:39.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-21T12:06:39.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:06:42.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-05-21T12:06:45.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-05-21T12:06:50.997 DEBUG:teuthology.orchestra.remote:timed out 2024-05-21T12:06:52.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-05-21T12:06:54.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-21T12:06:54.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:06:57.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-21T12:06:57.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:06:59.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-21T12:06:59.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:07:00.159 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-05-21T12:07:00.568 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-05-21T12:07:00.725 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-21T12:07:00.726 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-447.el9.x86_64"... 2024-05-21T12:07:00.726 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-05-21T12:07:00.741 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-447.el9.x86_64 2024-05-21T12:07:00.741 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T12:07:00.741 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T12:07:00.741 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T12:07:01.011 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-21T12:07:01.011 INFO:teuthology.task.kernel:Checking kernel version of host.c, want "5.14.0-447.el9.x86_64"... 2024-05-21T12:07:01.011 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-05-21T12:07:01.026 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-447.el9.x86_64 2024-05-21T12:07:01.026 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T12:07:01.026 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T12:07:01.026 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T12:07:01.743 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-21T12:07:01.743 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-21T12:07:01.815 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2024-05-21T12:07:01.841 DEBUG:teuthology.parallel:result is None 2024-05-21T12:07:02.027 INFO:teuthology.task.kernel:Enabling kdb on host.c... 2024-05-21T12:07:02.027 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-21T12:07:02.115 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-05-21T12:07:02.145 DEBUG:teuthology.parallel:result is None 2024-05-21T12:07:04.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-05-21T12:07:04.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:07:04.834 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-05-21T12:07:05.248 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-05-21T12:07:05.248 INFO:teuthology.task.kernel:Checking kernel version of client.3, want "5.14.0-447.el9.x86_64"... 2024-05-21T12:07:05.248 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-05-21T12:07:05.263 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-447.el9.x86_64 2024-05-21T12:07:05.263 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T12:07:05.263 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T12:07:05.263 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T12:07:06.264 INFO:teuthology.task.kernel:Enabling kdb on client.3... 2024-05-21T12:07:06.265 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-21T12:07:06.339 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-05-21T12:07:06.372 DEBUG:teuthology.parallel:result is None 2024-05-21T12:07:26.463 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-05-21T12:07:27.499 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-21T12:07:27.499 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-447.el9.x86_64"... 2024-05-21T12:07:27.499 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2024-05-21T12:07:27.514 INFO:teuthology.orchestra.run.smithi072.stdout:5.14.0-447.el9.x86_64 2024-05-21T12:07:27.515 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T12:07:27.515 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T12:07:27.515 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T12:07:28.516 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-21T12:07:28.516 DEBUG:teuthology.orchestra.run.smithi072:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-21T12:07:28.632 INFO:teuthology.orchestra.run.smithi072.stdout:ttyS1 2024-05-21T12:07:28.762 DEBUG:teuthology.parallel:result is None 2024-05-21T12:07:28.762 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-21T12:07:28.772 INFO:teuthology.task.internal:Creating test directory... 2024-05-21T12:07:28.772 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T12:07:28.776 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T12:07:28.777 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T12:07:28.781 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T12:07:28.801 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-21T12:07:28.849 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-21T12:07:28.856 INFO:teuthology.task.internal:Creating archive directory... 2024-05-21T12:07:28.857 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T12:07:28.859 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T12:07:28.862 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T12:07:28.864 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T12:07:29.195 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-21T12:07:29.202 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-21T12:07:29.203 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T12:07:29.205 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T12:07:29.207 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T12:07:29.210 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T12:07:29.238 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T12:07:29.241 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T12:07:29.246 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T12:07:29.251 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T12:07:29.253 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T12:07:29.258 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T12:07:29.261 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T12:07:29.271 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T12:07:29.273 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-21T12:07:29.280 INFO:teuthology.task.internal:Configuring sudo... 2024-05-21T12:07:29.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T12:07:29.302 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T12:07:29.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T12:07:29.307 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T12:07:29.336 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-21T12:07:29.344 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-21T12:07:29.345 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T12:07:29.373 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T12:07:29.376 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T12:07:29.380 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T12:07:29.397 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T12:07:29.477 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T12:07:29.529 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T12:07:29.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T12:07:29.661 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:07:29.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T12:07:29.730 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T12:07:29.773 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T12:07:29.823 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T12:07:29.909 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T12:07:29.952 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:07:29.952 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T12:07:30.031 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T12:07:30.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T12:07:30.134 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T12:07:30.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T12:07:30.260 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:07:30.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T12:07:30.326 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T12:07:30.373 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T12:07:30.428 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T12:07:30.517 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T12:07:30.557 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:07:30.557 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T12:07:30.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-05-21T12:07:30.627 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2024-05-21T12:07:30.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-05-21T12:07:30.632 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-05-21T12:07:30.677 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T12:07:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T12:07:30.685 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T12:07:30.714 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T12:07:31.228 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-21T12:07:31.235 INFO:teuthology.task.internal:Starting timer... 2024-05-21T12:07:31.235 INFO:teuthology.run_tasks:Running task pcp... 2024-05-21T12:07:31.246 INFO:teuthology.run_tasks:Running task selinux... 2024-05-21T12:07:31.255 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-21T12:07:31.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-05-21T12:07:31.303 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-05-21T12:07:31.306 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2024-05-21T12:07:31.394 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2024-05-21T12:07:31.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-05-21T12:07:31.486 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-05-21T12:07:31.488 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-05-21T12:07:31.564 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-05-21T12:07:31.566 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-21T12:07:31.566 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-05-21T12:07:31.619 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-05-21T12:07:31.619 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2024-05-21T12:07:31.642 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2024-05-21T12:07:31.643 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-05-21T12:07:31.669 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-05-21T12:07:31.670 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-05-21T12:07:31.705 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-05-21T12:07:31.705 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive'} 2024-05-21T12:07:31.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T12:07:31.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:07:31.736 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T12:07:31.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:07:31.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T12:07:31.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:07:31.793 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T12:07:31.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:07:31.823 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-21T12:07:31.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-05-21T12:07:31.855 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2024-05-21T12:07:31.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-05-21T12:07:31.914 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-05-21T12:07:31.941 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-21T12:07:31.953 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-21T12:07:31.964 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-21T12:07:31.965 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi072.front.sepia.ceph.com,smithi092.front.sepia.ceph.com,smithi137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-21T12:12:11.742 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com')] 2024-05-21T12:12:11.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-21T12:12:11.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:12:11.824 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-05-21T12:12:11.907 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-21T12:12:11.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-21T12:12:11.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:12:11.986 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-05-21T12:12:12.072 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-21T12:12:12.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-21T12:12:12.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:12:12.147 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-05-21T12:12:12.227 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-21T12:12:12.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-05-21T12:12:12.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T12:12:12.304 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-05-21T12:12:12.395 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-05-21T12:12:12.396 INFO:teuthology.run_tasks:Running task clock... 2024-05-21T12:12:12.409 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-21T12:12:12.410 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T12:12:12.410 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T12:12:12.413 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T12:12:12.413 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T12:12:12.415 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T12:12:12.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T12:12:12.418 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T12:12:12.418 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T12:12:12.445 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T12:12:12.447 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T12:12:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T12:12:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T12:12:12.463 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T12:12:12.463 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T12:12:12.467 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T12:12:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T12:12:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-05-21T12:12:12.489 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2024-05-21T12:12:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-05-21T12:12:12.499 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-05-21T12:12:12.501 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2024-05-21T12:12:12.502 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-05-21T12:12:12.507 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-05-21T12:12:12.514 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T12:12:12.516 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T12:12:12.518 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T12:12:12.518 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-05-21T12:12:12.528 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T12:12:12.530 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T12:12:12.533 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T12:12:12.533 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T12:12:12.547 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T12:12:12.586 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-05-21T12:12:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-21T12:12:12.589 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T12:12:12.589 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-05-21T12:12:12.589 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.589 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.589 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.589 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.590 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T12:12:12.591 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-21T12:12:12.591 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.591 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.591 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.591 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.591 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-05-21T12:12:12.594 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T12:12:12.594 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-05-21T12:12:12.594 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.594 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.595 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.595 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-05-21T12:12:12.607 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T12:12:12.608 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-05-21T12:12:12.608 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.608 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.608 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.608 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:12:12.608 INFO:teuthology.run_tasks:Running task install... 2024-05-21T12:12:12.617 DEBUG:teuthology.task.install:project ceph 2024-05-21T12:12:12.617 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0c764b6e22fbb51f59e469d9dd99895c152f73e'}} 2024-05-21T12:12:12.617 DEBUG:teuthology.task.install:config {'extra_packages': ['nvme-cli'], 'flavor': 'default', 'sha1': 'b0c764b6e22fbb51f59e469d9dd99895c152f73e'} 2024-05-21T12:12:12.617 INFO:teuthology.task.install:Using flavor: default 2024-05-21T12:12:12.623 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-21T12:12:12.623 INFO:teuthology.task.install:extra packages: ['nvme-cli'] 2024-05-21T12:12:12.623 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['nvme-cli'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0c764b6e22fbb51f59e469d9dd99895c152f73e', 'tag': None, 'wait_for_package': False} 2024-05-21T12:12:12.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:12:12.626 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['nvme-cli'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0c764b6e22fbb51f59e469d9dd99895c152f73e', 'tag': None, 'wait_for_package': False} 2024-05-21T12:12:12.626 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:12:12.627 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['nvme-cli'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0c764b6e22fbb51f59e469d9dd99895c152f73e', 'tag': None, 'wait_for_package': False} 2024-05-21T12:12:12.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:12:12.628 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['nvme-cli'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0c764b6e22fbb51f59e469d9dd99895c152f73e', 'tag': None, 'wait_for_package': False} 2024-05-21T12:12:12.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:12:12.773 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/ 2024-05-21T12:12:12.773 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3665.gb0c764b6 2024-05-21T12:12:12.778 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/ 2024-05-21T12:12:12.778 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3665.gb0c764b6 2024-05-21T12:12:12.796 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/ 2024-05-21T12:12:12.796 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3665.gb0c764b6 2024-05-21T12:12:12.883 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T12:12:12.883 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:12:12.883 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T12:12:12.910 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, nvme-cli on remote rpm x86_64 2024-05-21T12:12:12.910 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0c764b6e22fbb51f59e469d9dd99895c152f73e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T12:12:12.976 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/ 2024-05-21T12:12:12.977 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3665.gb0c764b6 2024-05-21T12:12:12.979 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T12:12:13.019 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T12:12:13.019 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:12:13.020 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T12:12:13.046 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, nvme-cli on remote rpm x86_64 2024-05-21T12:12:13.046 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0c764b6e22fbb51f59e469d9dd99895c152f73e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T12:12:13.055 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T12:12:13.055 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:12:13.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T12:12:13.056 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T12:12:13.083 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, nvme-cli on remote rpm x86_64 2024-05-21T12:12:13.083 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0c764b6e22fbb51f59e469d9dd99895c152f73e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T12:12:13.111 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T12:12:13.120 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2024-05-21T12:12:13.122 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2024-05-21T12:12:13.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T12:12:13.187 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T12:12:13.195 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T12:12:13.195 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:12:13.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T12:12:13.224 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, nvme-cli on remote rpm x86_64 2024-05-21T12:12:13.224 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0c764b6e22fbb51f59e469d9dd99895c152f73e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T12:12:13.225 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T12:12:13.251 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-05-21T12:12:13.253 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-05-21T12:12:13.290 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-05-21T12:12:13.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-05-21T12:12:13.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T12:12:13.346 INFO:teuthology.orchestra.run.smithi072.stdout:45 files removed 2024-05-21T12:12:13.373 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw 2024-05-21T12:12:13.374 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T12:12:13.441 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-05-21T12:12:13.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-05-21T12:12:13.477 INFO:teuthology.orchestra.run.smithi137.stdout:45 files removed 2024-05-21T12:12:13.504 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw 2024-05-21T12:12:13.513 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-05-21T12:12:13.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw 2024-05-21T12:12:13.677 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-05-21T12:12:13.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw 2024-05-21T12:12:13.974 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 257 kB/s | 84 kB 00:00 2024-05-21T12:12:14.302 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 171 kB/s | 84 kB 00:00 2024-05-21T12:12:14.383 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 139 kB/s | 84 kB 00:00 2024-05-21T12:12:14.467 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-05-21T12:12:14.468 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 177 kB/s | 84 kB 00:00 2024-05-21T12:12:14.800 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-05-21T12:12:14.819 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-05-21T12:12:14.885 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-21T12:12:14.909 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-21T12:12:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-21T12:12:15.301 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-05-21T12:12:15.310 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-05-21T12:12:17.118 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-21T12:12:17.220 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.1 MB 00:01 2024-05-21T12:12:17.520 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.1 MB 00:02 2024-05-21T12:12:18.202 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.1 MB 00:02 2024-05-21T12:12:20.124 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-21T12:12:20.278 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-05-21T12:12:22.927 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 19 MB 00:04 2024-05-21T12:12:25.787 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-05-21T12:12:25.792 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.2 MB 00:02 2024-05-21T12:12:27.169 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-05-21T12:12:27.229 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-21T12:12:27.352 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-05-21T12:12:28.466 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-21T12:12:28.928 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 16 kB 00:00 2024-05-21T12:12:30.180 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-21T12:12:33.568 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-21T12:12:33.575 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-21T12:12:35.372 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-21T12:12:35.683 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 2.5 MB/s | 21 MB 00:08 2024-05-21T12:12:35.927 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:12:35.945 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:12:35.945 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:12:35.945 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:12:35.945 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 11 M 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.4 M 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.1 M 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 5.4 M 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 21 M 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 25 k 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T12:12:35.946 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 747 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 501 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 5.4 M 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T12:12:35.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 45 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 136 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 162 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 320 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 300 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 99 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T12:12:35.948 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2024-05-21T12:12:35.949 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T12:12:35.949 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:12:35.949 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:12:35.949 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:12:35.949 INFO:teuthology.orchestra.run.smithi072.stdout:Install 30 Packages 2024-05-21T12:12:35.949 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 2 Packages 2024-05-21T12:12:35.949 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:12:35.950 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 60 M 2024-05-21T12:12:35.950 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:12:37.554 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 11 M 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.4 M 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.1 M 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T12:12:37.561 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 5.4 M 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 21 M 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 25 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 747 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 501 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T12:12:37.562 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 5.4 M 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 45 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 136 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 162 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 320 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 300 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 99 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T12:12:37.563 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:Install 30 Packages 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 2 Packages 2024-05-21T12:12:37.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:12:37.565 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 60 M 2024-05-21T12:12:37.565 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:12:37.820 INFO:teuthology.orchestra.run.smithi072.stdout:(1/32): ceph-base-19.0.0-3665.gb0c764b6.el9.x86 3.8 MB/s | 5.4 MB 00:01 2024-05-21T12:12:37.887 INFO:teuthology.orchestra.run.smithi072.stdout:(2/32): ceph-selinux-19.0.0-3665.gb0c764b6.el9. 377 kB/s | 25 kB 00:00 2024-05-21T12:12:38.112 INFO:teuthology.orchestra.run.smithi072.stdout:(3/32): libcephfs2-19.0.0-3665.gb0c764b6.el9.x8 3.2 MB/s | 747 kB 00:00 2024-05-21T12:12:38.213 INFO:teuthology.orchestra.run.smithi072.stdout:(4/32): libradosstriper1-19.0.0-3665.gb0c764b6. 4.9 MB/s | 501 kB 00:00 2024-05-21T12:12:38.477 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-05-21T12:12:39.398 INFO:teuthology.orchestra.run.smithi072.stdout:(5/32): librgw2-19.0.0-3665.gb0c764b6.el9.x86_6 4.5 MB/s | 5.4 MB 00:01 2024-05-21T12:12:39.465 INFO:teuthology.orchestra.run.smithi072.stdout:(6/32): python3-ceph-argparse-19.0.0-3665.gb0c7 677 kB/s | 45 kB 00:00 2024-05-21T12:12:39.549 INFO:teuthology.orchestra.run.smithi072.stdout:(7/32): python3-ceph-common-19.0.0-3665.gb0c764 1.6 MB/s | 136 kB 00:00 2024-05-21T12:12:39.625 INFO:teuthology.orchestra.run.smithi072.stdout:(8/32): python3-cephfs-19.0.0-3665.gb0c764b6.el 2.1 MB/s | 162 kB 00:00 2024-05-21T12:12:39.750 INFO:teuthology.orchestra.run.smithi072.stdout:(9/32): python3-rados-19.0.0-3665.gb0c764b6.el9 2.5 MB/s | 320 kB 00:00 2024-05-21T12:12:40.044 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-05-21T12:12:40.092 INFO:teuthology.orchestra.run.smithi072.stdout:(10/32): ceph-common-19.0.0-3665.gb0c764b6.el9. 5.8 MB/s | 21 MB 00:03 2024-05-21T12:12:40.118 INFO:teuthology.orchestra.run.smithi072.stdout:(11/32): python3-rbd-19.0.0-3665.gb0c764b6.el9. 817 kB/s | 300 kB 00:00 2024-05-21T12:12:40.144 INFO:teuthology.orchestra.run.smithi072.stdout:(12/32): python3-rgw-19.0.0-3665.gb0c764b6.el9. 1.9 MB/s | 99 kB 00:00 2024-05-21T12:12:40.328 INFO:teuthology.orchestra.run.smithi072.stdout:(13/32): ceph-radosgw-19.0.0-3665.gb0c764b6.el9 2.7 MB/s | 11 MB 00:03 2024-05-21T12:12:40.445 INFO:teuthology.orchestra.run.smithi072.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-05-21T12:12:40.470 INFO:teuthology.orchestra.run.smithi072.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 304 kB/s | 107 kB 00:00 2024-05-21T12:12:40.504 INFO:teuthology.orchestra.run.smithi072.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 445 kB/s | 160 kB 00:00 2024-05-21T12:12:40.555 INFO:teuthology.orchestra.run.smithi072.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 841 kB/s | 42 kB 00:00 2024-05-21T12:12:40.614 INFO:teuthology.orchestra.run.smithi072.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-05-21T12:12:40.672 INFO:teuthology.orchestra.run.smithi072.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-05-21T12:12:40.699 INFO:teuthology.orchestra.run.smithi072.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 420 kB/s | 36 kB 00:00 2024-05-21T12:12:40.734 INFO:teuthology.orchestra.run.smithi072.stdout:(21/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-05-21T12:12:40.759 INFO:teuthology.orchestra.run.smithi072.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 549 kB/s | 47 kB 00:00 2024-05-21T12:12:40.814 INFO:teuthology.orchestra.run.smithi137.stdout:(1/32): ceph-base-19.0.0-3665.gb0c764b6.el9.x86 1.9 MB/s | 5.4 MB 00:02 2024-05-21T12:12:40.864 INFO:teuthology.orchestra.run.smithi137.stdout:(2/32): ceph-selinux-19.0.0-3665.gb0c764b6.el9. 506 kB/s | 25 kB 00:00 2024-05-21T12:12:40.980 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-05-21T12:12:41.029 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-05-21T12:12:41.064 INFO:teuthology.orchestra.run.smithi137.stdout:(3/32): libcephfs2-19.0.0-3665.gb0c764b6.el9.x8 3.6 MB/s | 747 kB 00:00 2024-05-21T12:12:41.198 INFO:teuthology.orchestra.run.smithi137.stdout:(4/32): libradosstriper1-19.0.0-3665.gb0c764b6. 3.7 MB/s | 501 kB 00:00 2024-05-21T12:12:42.215 INFO:teuthology.orchestra.run.smithi137.stdout:(5/32): librgw2-19.0.0-3665.gb0c764b6.el9.x86_6 5.3 MB/s | 5.4 MB 00:01 2024-05-21T12:12:42.265 INFO:teuthology.orchestra.run.smithi137.stdout:(6/32): python3-ceph-argparse-19.0.0-3665.gb0c7 905 kB/s | 45 kB 00:00 2024-05-21T12:12:42.315 INFO:teuthology.orchestra.run.smithi137.stdout:(7/32): python3-ceph-common-19.0.0-3665.gb0c764 2.7 MB/s | 136 kB 00:00 2024-05-21T12:12:42.383 INFO:teuthology.orchestra.run.smithi137.stdout:(8/32): python3-cephfs-19.0.0-3665.gb0c764b6.el 2.4 MB/s | 162 kB 00:00 2024-05-21T12:12:42.474 INFO:teuthology.orchestra.run.smithi137.stdout:(9/32): python3-rados-19.0.0-3665.gb0c764b6.el9 3.4 MB/s | 320 kB 00:00 2024-05-21T12:12:42.566 INFO:teuthology.orchestra.run.smithi137.stdout:(10/32): python3-rbd-19.0.0-3665.gb0c764b6.el9. 3.2 MB/s | 300 kB 00:00 2024-05-21T12:12:42.617 INFO:teuthology.orchestra.run.smithi137.stdout:(11/32): python3-rgw-19.0.0-3665.gb0c764b6.el9. 1.9 MB/s | 99 kB 00:00 2024-05-21T12:12:42.801 INFO:teuthology.orchestra.run.smithi137.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 580 kB/s | 107 kB 00:00 2024-05-21T12:12:42.894 INFO:teuthology.orchestra.run.smithi137.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-05-21T12:12:42.901 INFO:teuthology.orchestra.run.smithi072.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 12 kB/s | 26 kB 00:02 2024-05-21T12:12:42.953 INFO:teuthology.orchestra.run.smithi137.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-05-21T12:12:43.045 INFO:teuthology.orchestra.run.smithi137.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-05-21T12:12:43.104 INFO:teuthology.orchestra.run.smithi137.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-21T12:12:43.171 INFO:teuthology.orchestra.run.smithi137.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 631 kB/s | 42 kB 00:00 2024-05-21T12:12:43.230 INFO:teuthology.orchestra.run.smithi137.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-21T12:12:43.289 INFO:teuthology.orchestra.run.smithi137.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 614 kB/s | 36 kB 00:00 2024-05-21T12:12:43.298 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:12:43.310 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:12:43.310 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:12:43.310 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:12:43.310 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:12:43.311 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 11 M 2024-05-21T12:12:43.311 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-05-21T12:12:43.311 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.4 M 2024-05-21T12:12:43.311 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.1 M 2024-05-21T12:12:43.311 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:12:43.311 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T12:12:43.312 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 5.4 M 2024-05-21T12:12:43.312 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 21 M 2024-05-21T12:12:43.312 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 25 k 2024-05-21T12:12:43.312 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T12:12:43.312 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T12:12:43.312 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T12:12:43.313 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 747 k 2024-05-21T12:12:43.313 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T12:12:43.313 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T12:12:43.313 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T12:12:43.313 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 501 k 2024-05-21T12:12:43.313 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T12:12:43.313 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 5.4 M 2024-05-21T12:12:43.314 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T12:12:43.314 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T12:12:43.314 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T12:12:43.314 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 45 k 2024-05-21T12:12:43.314 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 136 k 2024-05-21T12:12:43.314 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 162 k 2024-05-21T12:12:43.314 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T12:12:43.315 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 320 k 2024-05-21T12:12:43.315 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 300 k 2024-05-21T12:12:43.315 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 99 k 2024-05-21T12:12:43.315 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T12:12:43.315 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T12:12:43.315 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T12:12:43.316 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T12:12:43.316 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-05-21T12:12:43.316 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T12:12:43.316 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:12:43.316 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:12:43.316 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:12:43.316 INFO:teuthology.orchestra.run.smithi045.stdout:Install 30 Packages 2024-05-21T12:12:43.317 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 2 Packages 2024-05-21T12:12:43.317 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:12:43.318 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 60 M 2024-05-21T12:12:43.318 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:12:43.347 INFO:teuthology.orchestra.run.smithi137.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 816 kB/s | 47 kB 00:00 2024-05-21T12:12:43.698 INFO:teuthology.orchestra.run.smithi137.stdout:(21/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 883 kB/s | 309 kB 00:00 2024-05-21T12:12:43.998 INFO:teuthology.orchestra.run.smithi137.stdout:(22/32): ceph-common-19.0.0-3665.gb0c764b6.el9. 3.6 MB/s | 21 MB 00:05 2024-05-21T12:12:44.185 INFO:teuthology.orchestra.run.smithi072.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 89 kB/s | 309 kB 00:03 2024-05-21T12:12:44.210 INFO:teuthology.orchestra.run.smithi072.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 37 kB/s | 49 kB 00:01 2024-05-21T12:12:44.231 INFO:teuthology.orchestra.run.smithi137.stdout:(23/32): libarrow-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-05-21T12:12:44.257 INFO:teuthology.orchestra.run.smithi137.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-05-21T12:12:44.357 INFO:teuthology.orchestra.run.smithi137.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-05-21T12:12:44.474 INFO:teuthology.orchestra.run.smithi137.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 310 kB/s | 67 kB 00:00 2024-05-21T12:12:44.783 INFO:teuthology.orchestra.run.smithi137.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 839 kB 00:00 2024-05-21T12:12:44.798 INFO:teuthology.orchestra.run.smithi045.stdout:(1/32): ceph-base-19.0.0-3665.gb0c764b6.el9.x86 4.1 MB/s | 5.4 MB 00:01 2024-05-21T12:12:44.840 INFO:teuthology.orchestra.run.smithi045.stdout:(2/32): ceph-selinux-19.0.0-3665.gb0c764b6.el9. 606 kB/s | 25 kB 00:00 2024-05-21T12:12:44.916 INFO:teuthology.orchestra.run.smithi137.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 433 kB/s | 192 kB 00:00 2024-05-21T12:12:44.923 INFO:teuthology.orchestra.run.smithi045.stdout:(3/32): libcephfs2-19.0.0-3665.gb0c764b6.el9.x8 8.8 MB/s | 747 kB 00:00 2024-05-21T12:12:45.023 INFO:teuthology.orchestra.run.smithi045.stdout:(4/32): libradosstriper1-19.0.0-3665.gb0c764b6. 4.9 MB/s | 501 kB 00:00 2024-05-21T12:12:45.177 INFO:teuthology.orchestra.run.smithi072.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 70 kB/s | 67 kB 00:00 2024-05-21T12:12:45.258 INFO:teuthology.orchestra.run.smithi137.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-05-21T12:12:45.549 INFO:teuthology.orchestra.run.smithi045.stdout:(5/32): librgw2-19.0.0-3665.gb0c764b6.el9.x86_6 10 MB/s | 5.4 MB 00:00 2024-05-21T12:12:45.582 INFO:teuthology.orchestra.run.smithi045.stdout:(6/32): python3-ceph-argparse-19.0.0-3665.gb0c7 1.3 MB/s | 45 kB 00:00 2024-05-21T12:12:45.633 INFO:teuthology.orchestra.run.smithi045.stdout:(7/32): python3-ceph-common-19.0.0-3665.gb0c764 2.7 MB/s | 136 kB 00:00 2024-05-21T12:12:45.683 INFO:teuthology.orchestra.run.smithi045.stdout:(8/32): python3-cephfs-19.0.0-3665.gb0c764b6.el 3.2 MB/s | 162 kB 00:00 2024-05-21T12:12:45.733 INFO:teuthology.orchestra.run.smithi045.stdout:(9/32): python3-rados-19.0.0-3665.gb0c764b6.el9 6.3 MB/s | 320 kB 00:00 2024-05-21T12:12:45.791 INFO:teuthology.orchestra.run.smithi045.stdout:(10/32): python3-rbd-19.0.0-3665.gb0c764b6.el9. 5.0 MB/s | 300 kB 00:00 2024-05-21T12:12:45.817 INFO:teuthology.orchestra.run.smithi137.stdout:(30/32): librados2-19.0.0-3665.gb0c764b6.el9.x8 3.8 MB/s | 3.4 MB 00:00 2024-05-21T12:12:45.834 INFO:teuthology.orchestra.run.smithi045.stdout:(11/32): python3-rgw-19.0.0-3665.gb0c764b6.el9. 2.3 MB/s | 99 kB 00:00 2024-05-21T12:12:45.900 INFO:teuthology.orchestra.run.smithi137.stdout:(31/32): librbd1-19.0.0-3665.gb0c764b6.el9.x86_ 4.9 MB/s | 3.1 MB 00:00 2024-05-21T12:12:46.009 INFO:teuthology.orchestra.run.smithi137.stdout:(32/32): ceph-radosgw-19.0.0-3665.gb0c764b6.el9 1.3 MB/s | 11 MB 00:07 2024-05-21T12:12:46.012 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:12:46.012 INFO:teuthology.orchestra.run.smithi137.stdout:Total 7.1 MB/s | 60 MB 00:08 2024-05-21T12:12:46.025 INFO:teuthology.orchestra.run.smithi045.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 558 kB/s | 107 kB 00:00 2024-05-21T12:12:46.095 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-21T12:12:46.121 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:12:46.142 INFO:teuthology.orchestra.run.smithi045.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-21T12:12:46.157 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:12:46.157 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:12:46.201 INFO:teuthology.orchestra.run.smithi045.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-05-21T12:12:46.376 INFO:teuthology.orchestra.run.smithi045.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-05-21T12:12:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-05-21T12:12:46.660 INFO:teuthology.orchestra.run.smithi045.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 230 kB/s | 42 kB 00:00 2024-05-21T12:12:46.744 INFO:teuthology.orchestra.run.smithi045.stdout:(18/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-21T12:12:46.776 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:12:46.776 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:12:46.802 INFO:teuthology.orchestra.run.smithi045.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 616 kB/s | 36 kB 00:00 2024-05-21T12:12:46.861 INFO:teuthology.orchestra.run.smithi045.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-05-21T12:12:47.153 INFO:teuthology.orchestra.run.smithi045.stdout:(21/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.0 MB/s | 309 kB 00:00 2024-05-21T12:12:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:(22/32): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-05-21T12:12:47.703 INFO:teuthology.orchestra.run.smithi045.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 392 kB/s | 26 kB 00:00 2024-05-21T12:12:47.762 INFO:teuthology.orchestra.run.smithi045.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-05-21T12:12:47.829 INFO:teuthology.orchestra.run.smithi045.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-21T12:12:47.929 INFO:teuthology.orchestra.run.smithi045.stdout:(26/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-05-21T12:12:47.987 INFO:teuthology.orchestra.run.smithi045.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-21T12:12:48.138 INFO:teuthology.orchestra.run.smithi045.stdout:(28/32): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-21T12:12:48.201 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:12:48.287 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:12:48.291 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:12:48.291 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 11 M 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.4 M 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.1 M 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 5.4 M 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 21 M 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 25 k 2024-05-21T12:12:48.292 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 747 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 501 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 5.4 M 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T12:12:48.293 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 45 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 136 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 162 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 320 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 300 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 99 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T12:12:48.294 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:12:48.295 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:12:48.295 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:12:48.295 INFO:teuthology.orchestra.run.smithi092.stdout:Install 30 Packages 2024-05-21T12:12:48.295 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-05-21T12:12:48.295 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:12:48.295 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-21T12:12:48.296 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 60 M 2024-05-21T12:12:48.296 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:12:48.323 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-21T12:12:48.348 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-21T12:12:48.370 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-21T12:12:48.394 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-21T12:12:48.430 INFO:teuthology.orchestra.run.smithi045.stdout:(29/32): librados2-19.0.0-3665.gb0c764b6.el9.x8 12 MB/s | 3.4 MB 00:00 2024-05-21T12:12:48.569 INFO:teuthology.orchestra.run.smithi072.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 192 kB/s | 839 kB 00:04 2024-05-21T12:12:48.589 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-21T12:12:48.630 INFO:teuthology.orchestra.run.smithi045.stdout:(30/32): librbd1-19.0.0-3665.gb0c764b6.el9.x86_ 16 MB/s | 3.1 MB 00:00 2024-05-21T12:12:48.643 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-21T12:12:49.039 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-21T12:12:49.090 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:12:49.182 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:12:49.268 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 8/34 2024-05-21T12:12:49.300 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 9/34 2024-05-21T12:12:49.346 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 9/34 2024-05-21T12:12:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-21T12:12:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9. 11/34 2024-05-21T12:12:49.455 INFO:teuthology.orchestra.run.smithi092.stdout:(1/32): ceph-base-19.0.0-3665.gb0c764b6.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-05-21T12:12:49.487 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 12/34 2024-05-21T12:12:49.511 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 13/34 2024-05-21T12:12:49.513 INFO:teuthology.orchestra.run.smithi092.stdout:(2/32): ceph-selinux-19.0.0-3665.gb0c764b6.el9. 432 kB/s | 25 kB 00:00 2024-05-21T12:12:49.575 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 13/34 2024-05-21T12:12:49.578 INFO:teuthology.orchestra.run.smithi072.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 44 kB/s | 192 kB 00:04 2024-05-21T12:12:49.610 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-21T12:12:49.655 INFO:teuthology.orchestra.run.smithi092.stdout:(3/32): libcephfs2-19.0.0-3665.gb0c764b6.el9.x8 5.2 MB/s | 747 kB 00:00 2024-05-21T12:12:49.717 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-21T12:12:49.753 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-21T12:12:49.793 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-21T12:12:49.805 INFO:teuthology.orchestra.run.smithi092.stdout:(4/32): libradosstriper1-19.0.0-3665.gb0c764b6. 3.3 MB/s | 501 kB 00:00 2024-05-21T12:12:49.834 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-21T12:12:49.863 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-21T12:12:49.895 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-21T12:12:50.164 INFO:teuthology.orchestra.run.smithi092.stdout:(5/32): ceph-radosgw-19.0.0-3665.gb0c764b6.el9. 6.5 MB/s | 11 MB 00:01 2024-05-21T12:12:50.216 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-21T12:12:50.239 INFO:teuthology.orchestra.run.smithi092.stdout:(6/32): python3-ceph-argparse-19.0.0-3665.gb0c7 602 kB/s | 45 kB 00:00 2024-05-21T12:12:50.250 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 22/34 2024-05-21T12:12:50.305 INFO:teuthology.orchestra.run.smithi045.stdout:(31/32): ceph-radosgw-19.0.0-3665.gb0c764b6.el9 1.6 MB/s | 11 MB 00:06 2024-05-21T12:12:50.323 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 22/34 2024-05-21T12:12:50.364 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 23/34 2024-05-21T12:12:50.373 INFO:teuthology.orchestra.run.smithi092.stdout:(7/32): python3-ceph-common-19.0.0-3665.gb0c764 1.0 MB/s | 136 kB 00:00 2024-05-21T12:12:50.387 INFO:teuthology.orchestra.run.smithi072.stdout:(29/32): librados2-19.0.0-3665.gb0c764b6.el9.x8 4.2 MB/s | 3.4 MB 00:00 2024-05-21T12:12:50.400 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-21T12:12:50.482 INFO:teuthology.orchestra.run.smithi092.stdout:(8/32): python3-cephfs-19.0.0-3665.gb0c764b6.el 1.5 MB/s | 162 kB 00:00 2024-05-21T12:12:50.690 INFO:teuthology.orchestra.run.smithi092.stdout:(9/32): librgw2-19.0.0-3665.gb0c764b6.el9.x86_6 6.1 MB/s | 5.4 MB 00:00 2024-05-21T12:12:50.724 INFO:teuthology.orchestra.run.smithi092.stdout:(10/32): python3-rados-19.0.0-3665.gb0c764b6.el 1.3 MB/s | 320 kB 00:00 2024-05-21T12:12:50.758 INFO:teuthology.orchestra.run.smithi092.stdout:(11/32): python3-rbd-19.0.0-3665.gb0c764b6.el9. 4.4 MB/s | 300 kB 00:00 2024-05-21T12:12:50.799 INFO:teuthology.orchestra.run.smithi092.stdout:(12/32): python3-rgw-19.0.0-3665.gb0c764b6.el9. 1.3 MB/s | 99 kB 00:00 2024-05-21T12:12:50.854 INFO:teuthology.orchestra.run.smithi072.stdout:(30/32): librbd1-19.0.0-3665.gb0c764b6.el9.x86_ 6.7 MB/s | 3.1 MB 00:00 2024-05-21T12:12:50.878 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-21T12:12:50.911 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 26/34 2024-05-21T12:12:50.982 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 26/34 2024-05-21T12:12:50.991 INFO:teuthology.orchestra.run.smithi092.stdout:(13/32): ceph-common-19.0.0-3665.gb0c764b6.el9. 8.6 MB/s | 21 MB 00:02 2024-05-21T12:12:51.050 INFO:teuthology.orchestra.run.smithi092.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 365 kB/s | 107 kB 00:00 2024-05-21T12:12:51.073 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 27/34 2024-05-21T12:12:51.105 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x8 28/34 2024-05-21T12:12:51.133 INFO:teuthology.orchestra.run.smithi092.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-05-21T12:12:51.175 INFO:teuthology.orchestra.run.smithi092.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 426 kB/s | 160 kB 00:00 2024-05-21T12:12:51.209 INFO:teuthology.orchestra.run.smithi092.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-05-21T12:12:51.226 INFO:teuthology.orchestra.run.smithi092.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 840 kB/s | 42 kB 00:00 2024-05-21T12:12:51.268 INFO:teuthology.orchestra.run.smithi092.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-21T12:12:51.285 INFO:teuthology.orchestra.run.smithi092.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 611 kB/s | 36 kB 00:00 2024-05-21T12:12:51.319 INFO:teuthology.orchestra.run.smithi092.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-21T12:12:51.344 INFO:teuthology.orchestra.run.smithi092.stdout:(22/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.4 MB/s | 292 kB 00:00 2024-05-21T12:12:51.602 INFO:teuthology.orchestra.run.smithi092.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 125 kB/s | 26 kB 00:00 2024-05-21T12:12:51.627 INFO:teuthology.orchestra.run.smithi092.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 902 kB/s | 309 kB 00:00 2024-05-21T12:12:51.670 INFO:teuthology.orchestra.run.smithi092.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 728 kB/s | 49 kB 00:00 2024-05-21T12:12:51.695 INFO:teuthology.orchestra.run.smithi092.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-21T12:12:51.787 INFO:teuthology.orchestra.run.smithi092.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-21T12:12:51.812 INFO:teuthology.orchestra.run.smithi092.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-21T12:12:51.904 INFO:teuthology.orchestra.run.smithi092.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-21T12:12:52.079 INFO:teuthology.orchestra.run.smithi092.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-05-21T12:12:52.238 INFO:teuthology.orchestra.run.smithi092.stdout:(31/32): librados2-19.0.0-3665.gb0c764b6.el9.x8 8.0 MB/s | 3.4 MB 00:00 2024-05-21T12:12:52.288 INFO:teuthology.orchestra.run.smithi092.stdout:(32/32): librbd1-19.0.0-3665.gb0c764b6.el9.x86_ 8.2 MB/s | 3.1 MB 00:00 2024-05-21T12:12:52.292 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:12:52.292 INFO:teuthology.orchestra.run.smithi092.stdout:Total 15 MB/s | 60 MB 00:03 2024-05-21T12:12:52.390 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:12:52.424 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:12:52.424 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:12:53.032 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:12:53.033 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:12:53.386 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:12:53.437 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:12:53.555 INFO:teuthology.orchestra.run.smithi045.stdout:(32/32): ceph-common-19.0.0-3665.gb0c764b6.el9. 2.1 MB/s | 21 MB 00:10 2024-05-21T12:12:53.559 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:12:53.559 INFO:teuthology.orchestra.run.smithi045.stdout:Total 5.9 MB/s | 60 MB 00:10 2024-05-21T12:12:53.663 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:12:53.704 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:12:53.704 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:12:54.002 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:12:54.050 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 30/34 2024-05-21T12:12:54.129 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 30/34 2024-05-21T12:12:54.129 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T12:12:54.129 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T12:12:54.130 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:12:54.160 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:12:54.351 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:12:54.353 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:12:54.436 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:12:54.522 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-21T12:12:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-21T12:12:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-21T12:12:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-21T12:12:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-21T12:12:54.843 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-21T12:12:54.896 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-21T12:12:55.272 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-21T12:12:55.337 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:12:55.418 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:12:55.507 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 8/34 2024-05-21T12:12:55.538 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 9/34 2024-05-21T12:12:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 9/34 2024-05-21T12:12:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-21T12:12:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9. 11/34 2024-05-21T12:12:55.749 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 12/34 2024-05-21T12:12:55.783 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 13/34 2024-05-21T12:12:55.837 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 13/34 2024-05-21T12:12:55.872 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-21T12:12:55.913 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:12:55.978 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-21T12:12:55.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-21T12:12:56.015 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-21T12:12:56.027 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-21T12:12:56.057 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-21T12:12:56.060 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-21T12:12:56.083 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-21T12:12:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-21T12:12:56.104 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-21T12:12:56.117 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-21T12:12:56.149 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-21T12:12:56.302 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-21T12:12:56.355 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-21T12:12:56.475 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-21T12:12:56.521 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 22/34 2024-05-21T12:12:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 22/34 2024-05-21T12:12:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 23/34 2024-05-21T12:12:56.686 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-21T12:12:56.714 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-21T12:12:56.820 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:12:56.927 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:12:57.019 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 8/34 2024-05-21T12:12:57.055 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 9/34 2024-05-21T12:12:57.118 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 9/34 2024-05-21T12:12:57.155 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-21T12:12:57.172 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-21T12:12:57.199 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9. 11/34 2024-05-21T12:12:57.207 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 26/34 2024-05-21T12:12:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 12/34 2024-05-21T12:12:57.278 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 26/34 2024-05-21T12:12:57.290 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 13/34 2024-05-21T12:12:57.346 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 27/34 2024-05-21T12:12:57.355 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 13/34 2024-05-21T12:12:57.377 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x8 28/34 2024-05-21T12:12:57.390 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-21T12:12:57.500 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-21T12:12:57.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-21T12:12:57.575 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-21T12:12:57.607 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-21T12:12:57.637 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-21T12:12:57.669 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-21T12:12:57.999 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-21T12:12:58.041 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 22/34 2024-05-21T12:12:58.132 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 22/34 2024-05-21T12:12:58.162 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 23/34 2024-05-21T12:12:58.201 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-21T12:12:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-21T12:12:58.734 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 26/34 2024-05-21T12:12:58.805 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 26/34 2024-05-21T12:12:58.878 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 27/34 2024-05-21T12:12:58.912 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x8 28/34 2024-05-21T12:12:59.371 INFO:teuthology.orchestra.run.smithi072.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 150 kB/s | 1.6 MB 00:10 2024-05-21T12:12:59.686 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:12:59.723 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:13:00.287 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:13:00.327 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 30/34 2024-05-21T12:13:00.420 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 30/34 2024-05-21T12:13:00.421 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T12:13:00.421 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T12:13:00.421 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:00.453 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:01.397 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:13:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:13:02.026 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:13:02.070 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 30/34 2024-05-21T12:13:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 30/34 2024-05-21T12:13:02.162 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T12:13:02.162 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T12:13:02.162 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:02.197 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:03.429 INFO:teuthology.orchestra.run.smithi072.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 199 kB/s | 4.4 MB 00:22 2024-05-21T12:13:03.433 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:03.433 INFO:teuthology.orchestra.run.smithi072.stdout:Total 2.2 MB/s | 60 MB 00:27 2024-05-21T12:13:03.536 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:13:03.573 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:13:03.573 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:13:03.887 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:03.888 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-05-21T12:13:03.888 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-05-21T12:13:03.888 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-05-21T12:13:03.888 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-05-21T12:13:03.888 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-05-21T12:13:03.888 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-05-21T12:13:03.888 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-05-21T12:13:03.888 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:04.216 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:13:04.217 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:13:04.879 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/34 2024-05-21T12:13:04.904 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/34 2024-05-21T12:13:04.905 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:04.905 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T12:13:04.905 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T12:13:04.905 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T12:13:04.905 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:04.906 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-21T12:13:04.989 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-21T12:13:04.990 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:06.041 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 5/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 6/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9. 8/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x8 9/34 2024-05-21T12:13:06.121 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 10/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 11/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 12/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 13/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-21T12:13:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-21T12:13:06.123 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-21T12:13:06.123 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-21T12:13:06.123 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-21T12:13:06.123 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-21T12:13:06.123 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-21T12:13:06.124 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-21T12:13:06.124 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-21T12:13:06.124 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-21T12:13:06.124 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:06.124 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-21T12:13:06.124 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/34 2024-05-21T12:13:06.154 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-21T12:13:06.188 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-21T12:13:06.213 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-21T12:13:06.236 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-21T12:13:06.262 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-21T12:13:06.447 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-21T12:13:06.502 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-21T12:13:06.852 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:06.852 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:06.852 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-05-21T12:13:06.852 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.852 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.852 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.853 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:06.854 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T12:13:06.855 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T12:13:06.855 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T12:13:06.855 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T12:13:06.855 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T12:13:06.855 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:06.855 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:13:06.897 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-21T12:13:07.187 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:13:07.188 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-test 2024-05-21T12:13:07.280 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:13:07.367 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 8/34 2024-05-21T12:13:07.390 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 9/34 2024-05-21T12:13:07.459 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 9/34 2024-05-21T12:13:07.491 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-21T12:13:07.538 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9. 11/34 2024-05-21T12:13:07.604 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 12/34 2024-05-21T12:13:07.637 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 13/34 2024-05-21T12:13:07.694 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 13/34 2024-05-21T12:13:07.724 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:32 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:13:07.732 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-21T12:13:07.844 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:13:07.845 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:07.845 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:13:07.845 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:07.845 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:13:07.845 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 49 M 2024-05-21T12:13:07.845 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:13:07.846 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 163 k 2024-05-21T12:13:07.846 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T12:13:07.846 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T12:13:07.846 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:07.846 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:13:07.846 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:07.846 INFO:teuthology.orchestra.run.smithi137.stdout:Install 4 Packages 2024-05-21T12:13:07.846 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:07.847 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 50 M 2024-05-21T12:13:07.847 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 211 M 2024-05-21T12:13:07.847 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:13:07.847 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-21T12:13:07.888 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-21T12:13:07.927 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-21T12:13:07.966 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-21T12:13:08.003 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-21T12:13:08.037 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-21T12:13:08.243 INFO:teuthology.orchestra.run.smithi137.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-05-21T12:13:08.294 INFO:teuthology.orchestra.run.smithi137.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-21T12:13:08.336 INFO:teuthology.orchestra.run.smithi137.stdout:(3/4): libcephsqlite-19.0.0-3665.gb0c764b6.el9. 439 kB/s | 163 kB 00:00 2024-05-21T12:13:08.348 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-21T12:13:08.409 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 22/34 2024-05-21T12:13:08.510 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 22/34 2024-05-21T12:13:08.531 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 23/34 2024-05-21T12:13:08.578 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-21T12:13:09.057 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-21T12:13:09.082 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 26/34 2024-05-21T12:13:09.154 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 26/34 2024-05-21T12:13:09.224 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 27/34 2024-05-21T12:13:09.262 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x8 28/34 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-05-21T12:13:10.137 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:10.902 INFO:teuthology.orchestra.run.smithi137.stdout:(4/4): ceph-test-19.0.0-3665.gb0c764b6.el9.x86_ 17 MB/s | 49 MB 00:02 2024-05-21T12:13:10.904 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:10.905 INFO:teuthology.orchestra.run.smithi137.stdout:Total 16 MB/s | 50 MB 00:03 2024-05-21T12:13:10.933 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:13:10.953 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:13:10.953 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:13:11.139 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/34 2024-05-21T12:13:11.164 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/34 2024-05-21T12:13:11.164 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:11.164 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T12:13:11.164 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T12:13:11.164 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T12:13:11.165 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:11.166 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-21T12:13:11.249 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-21T12:13:11.250 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:11.333 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:13:11.333 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:13:11.605 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:13:11.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:13:11.859 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:13:11.937 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-21T12:13:11.976 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-21T12:13:12.004 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/4 2024-05-21T12:13:12.180 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:12.180 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-05-21T12:13:12.180 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-05-21T12:13:12.180 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-05-21T12:13:12.180 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-05-21T12:13:12.180 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-05-21T12:13:12.180 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-05-21T12:13:12.181 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-05-21T12:13:12.181 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:12.202 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 29/34 2024-05-21T12:13:12.252 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 30/34 2024-05-21T12:13:12.341 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 30/34 2024-05-21T12:13:12.342 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T12:13:12.342 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T12:13:12.342 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:12.371 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/34 2024-05-21T12:13:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/34 2024-05-21T12:13:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/34 2024-05-21T12:13:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 5/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 6/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9. 8/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x8 9/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 10/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 11/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 12/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 13/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-21T12:13:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-21T12:13:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-21T12:13:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-21T12:13:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-21T12:13:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-21T12:13:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-21T12:13:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/34 2024-05-21T12:13:13.198 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/34 2024-05-21T12:13:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/34 2024-05-21T12:13:13.223 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:13.223 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T12:13:13.224 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T12:13:13.224 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T12:13:13.224 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-21T12:13:13.266 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:13.266 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:13.266 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-05-21T12:13:13.266 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.267 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T12:13:13.268 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:13.269 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:13:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-21T12:13:13.308 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:13.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-test 2024-05-21T12:13:14.140 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:28 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:13:14.262 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 49 M 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 163 k 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:13:14.263 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:14.264 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-05-21T12:13:14.264 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:14.264 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 50 M 2024-05-21T12:13:14.264 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 211 M 2024-05-21T12:13:14.264 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:13:14.504 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 5/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 6/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9. 8/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x8 9/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 10/34 2024-05-21T12:13:14.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 11/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 12/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 13/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-21T12:13:14.506 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-21T12:13:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/34 2024-05-21T12:13:14.620 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-05-21T12:13:14.687 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-05-21T12:13:14.745 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): libcephsqlite-19.0.0-3665.gb0c764b6.el9. 446 kB/s | 163 kB 00:00 2024-05-21T12:13:15.420 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.421 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T12:13:15.422 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T12:13:15.423 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T12:13:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:15.424 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:13:15.758 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-test 2024-05-21T12:13:16.315 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:36 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:13:16.442 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:13:16.442 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:16.442 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 49 M 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 163 k 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-05-21T12:13:16.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:16.444 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 50 M 2024-05-21T12:13:16.444 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 211 M 2024-05-21T12:13:16.444 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:13:16.644 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/4 2024-05-21T12:13:16.720 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/4 2024-05-21T12:13:16.923 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): libcephsqlite-19.0.0-3665.gb0c764b6.el9. 397 kB/s | 163 kB 00:00 2024-05-21T12:13:17.162 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 466 kB/s | 305 kB 00:00 2024-05-21T12:13:17.270 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 182 kB/s | 64 kB 00:00 2024-05-21T12:13:19.104 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/4 2024-05-21T12:13:19.105 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/4 2024-05-21T12:13:19.105 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/4 2024-05-21T12:13:19.105 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-21T12:13:19.120 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): ceph-test-19.0.0-3665.gb0c764b6.el9.x86_ 10 MB/s | 49 MB 00:04 2024-05-21T12:13:19.122 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:19.122 INFO:teuthology.orchestra.run.smithi092.stdout:Total 10 MB/s | 50 MB 00:04 2024-05-21T12:13:19.149 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:13:19.170 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:13:19.170 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:13:19.399 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-21T12:13:19.399 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:19.399 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:13:19.399 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:19.399 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:19.399 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T12:13:19.400 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T12:13:19.400 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:19.400 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:13:19.555 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:13:19.555 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:13:19.679 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph 2024-05-21T12:13:20.080 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:13:20.164 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-21T12:13:20.210 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:45 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:13:20.211 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-21T12:13:20.238 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/4 2024-05-21T12:13:20.336 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:13:20.338 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:20.338 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 6.6 k 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 2.4 M 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 1.5 M 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 274 k 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 4.8 M 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 17 M 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T12:13:20.339 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T12:13:20.340 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T12:13:20.341 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:20.342 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:13:20.342 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:20.342 INFO:teuthology.orchestra.run.smithi137.stdout:Install 36 Packages 2024-05-21T12:13:20.342 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:20.343 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 30 M 2024-05-21T12:13:20.343 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 108 M 2024-05-21T12:13:20.343 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:13:20.995 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): ceph-test-19.0.0-3665.gb0c764b6.el9.x86_ 11 MB/s | 49 MB 00:04 2024-05-21T12:13:20.997 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:20.998 INFO:teuthology.orchestra.run.smithi045.stdout:Total 11 MB/s | 50 MB 00:04 2024-05-21T12:13:21.026 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:13:21.049 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:13:21.049 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:13:21.096 INFO:teuthology.orchestra.run.smithi137.stdout:(1/36): ceph-19.0.0-3665.gb0c764b6.el9.x86_64.r 42 kB/s | 6.6 kB 00:00 2024-05-21T12:13:21.362 INFO:teuthology.orchestra.run.smithi137.stdout:(2/36): ceph-mgr-19.0.0-3665.gb0c764b6.el9.x86_ 3.6 MB/s | 1.5 MB 00:00 2024-05-21T12:13:21.452 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:13:21.453 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:13:21.530 INFO:teuthology.orchestra.run.smithi137.stdout:(3/36): ceph-mon-19.0.0-3665.gb0c764b6.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-05-21T12:13:21.605 INFO:teuthology.orchestra.run.smithi137.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3665.gb0c7 3.6 MB/s | 274 kB 00:00 2024-05-21T12:13:21.747 INFO:teuthology.orchestra.run.smithi137.stdout:(5/36): ceph-mds-19.0.0-3665.gb0c764b6.el9.x86_ 2.9 MB/s | 2.4 MB 00:00 2024-05-21T12:13:21.889 INFO:teuthology.orchestra.run.smithi137.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 893 kB/s | 253 kB 00:00 2024-05-21T12:13:21.972 INFO:teuthology.orchestra.run.smithi137.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-05-21T12:13:22.001 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:13:22.047 INFO:teuthology.orchestra.run.smithi137.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-05-21T12:13:22.089 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-21T12:13:22.136 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-21T12:13:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/4 2024-05-21T12:13:22.256 INFO:teuthology.orchestra.run.smithi137.stdout:(9/36): ceph-osd-19.0.0-3665.gb0c764b6.el9.x86_ 19 MB/s | 17 MB 00:00 2024-05-21T12:13:22.290 INFO:teuthology.orchestra.run.smithi137.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 515 kB/s | 125 kB 00:00 2024-05-21T12:13:22.390 INFO:teuthology.orchestra.run.smithi137.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 1.9 MB/s | 1.2 MB 00:00 2024-05-21T12:13:22.507 INFO:teuthology.orchestra.run.smithi137.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 857 kB/s | 215 kB 00:00 2024-05-21T12:13:23.135 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:23.135 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2024-05-21T12:13:23.135 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2024-05-21T12:13:23.135 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2024-05-21T12:13:23.136 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2024-05-21T12:13:23.136 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2024-05-21T12:13:23.136 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /root 2024-05-21T12:13:23.136 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2024-05-21T12:13:23.136 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:23.224 INFO:teuthology.orchestra.run.smithi137.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 48 kB/s | 35 kB 00:00 2024-05-21T12:13:23.241 INFO:teuthology.orchestra.run.smithi137.stdout:(14/36): lua-5.4.4-4.el9.x86_64.rpm 198 kB/s | 188 kB 00:00 2024-05-21T12:13:23.333 INFO:teuthology.orchestra.run.smithi137.stdout:(15/36): python3-mako-1.1.4-6.el9.noarch.rpm 183 kB/s | 172 kB 00:00 2024-05-21T12:13:23.383 INFO:teuthology.orchestra.run.smithi137.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 448 kB/s | 22 kB 00:00 2024-05-21T12:13:23.842 INFO:teuthology.orchestra.run.smithi137.stdout:(17/36): luarocks-3.9.2-5.el9.noarch.rpm 330 kB/s | 151 kB 00:00 2024-05-21T12:13:23.942 INFO:teuthology.orchestra.run.smithi137.stdout:(18/36): python3-pytz-2021.1-5.el9.noarch.rpm 71 kB/s | 51 kB 00:00 2024-05-21T12:13:23.967 INFO:teuthology.orchestra.run.smithi137.stdout:(19/36): python3-toml-0.10.2-6.el9.noarch.rpm 57 kB/s | 42 kB 00:00 2024-05-21T12:13:24.034 INFO:teuthology.orchestra.run.smithi137.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 226 kB/s | 43 kB 00:00 2024-05-21T12:13:24.124 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/34 2024-05-21T12:13:24.143 INFO:teuthology.orchestra.run.smithi137.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 860 kB/s | 172 kB 00:00 2024-05-21T12:13:24.150 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/34 2024-05-21T12:13:24.150 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:24.150 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T12:13:24.150 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T12:13:24.150 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T12:13:24.150 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:24.152 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-21T12:13:24.247 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-21T12:13:24.247 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:24.310 INFO:teuthology.orchestra.run.smithi137.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 39 kB/s | 11 kB 00:00 2024-05-21T12:13:24.335 INFO:teuthology.orchestra.run.smithi137.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 92 kB/s | 18 kB 00:00 2024-05-21T12:13:24.360 INFO:teuthology.orchestra.run.smithi137.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 913 kB/s | 358 kB 00:00 2024-05-21T12:13:24.468 INFO:teuthology.orchestra.run.smithi137.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 173 kB/s | 23 kB 00:00 2024-05-21T12:13:24.502 INFO:teuthology.orchestra.run.smithi137.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 116 kB/s | 19 kB 00:00 2024-05-21T12:13:24.552 INFO:teuthology.orchestra.run.smithi137.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 102 kB/s | 20 kB 00:00 2024-05-21T12:13:24.569 INFO:teuthology.orchestra.run.smithi137.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 460 kB/s | 46 kB 00:00 2024-05-21T12:13:24.628 INFO:teuthology.orchestra.run.smithi137.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 628 kB/s | 79 kB 00:00 2024-05-21T12:13:24.737 INFO:teuthology.orchestra.run.smithi137.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-05-21T12:13:24.762 INFO:teuthology.orchestra.run.smithi137.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 85 kB/s | 16 kB 00:00 2024-05-21T12:13:24.787 INFO:teuthology.orchestra.run.smithi137.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 566 kB/s | 90 kB 00:00 2024-05-21T12:13:24.846 INFO:teuthology.orchestra.run.smithi137.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 329 kB/s | 36 kB 00:00 2024-05-21T12:13:24.938 INFO:teuthology.orchestra.run.smithi137.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-05-21T12:13:24.975 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/4 2024-05-21T12:13:25.013 INFO:teuthology.orchestra.run.smithi137.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 120 kB/s | 20 kB 00:00 2024-05-21T12:13:25.053 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/4 2024-05-21T12:13:25.121 INFO:teuthology.orchestra.run.smithi137.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.2 MB/s | 427 kB 00:00 2024-05-21T12:13:25.125 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:25.126 INFO:teuthology.orchestra.run.smithi137.stdout:Total 6.3 MB/s | 30 MB 00:04 2024-05-21T12:13:25.271 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:13:25.311 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:13:25.311 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 5/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_6 6/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 7/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9. 8/34 2024-05-21T12:13:25.744 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x8 9/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 10/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 11/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 12/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 13/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-21T12:13:25.745 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 31/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-21T12:13:25.746 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/34 2024-05-21T12:13:25.754 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:13:25.754 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:13:26.535 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-21T12:13:26.535 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:26.535 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-05-21T12:13:26.535 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.535 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T12:13:26.536 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.537 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.538 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:26.538 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T12:13:26.538 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T12:13:26.538 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T12:13:26.538 INFO:teuthology.orchestra.run.smithi072.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T12:13:26.538 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T12:13:26.538 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:26.538 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:13:26.941 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-test 2024-05-21T12:13:27.020 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:13:27.082 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-21T12:13:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/4 2024-05-21T12:13:27.119 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-21T12:13:27.151 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-21T12:13:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/4 2024-05-21T12:13:27.222 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-21T12:13:27.275 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/36 2024-05-21T12:13:27.308 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-21T12:13:27.400 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-21T12:13:27.478 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-21T12:13:27.479 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:54 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:13:27.522 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-21T12:13:27.554 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-21T12:13:27.562 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/4 2024-05-21T12:13:27.562 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/4 2024-05-21T12:13:27.562 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/4 2024-05-21T12:13:27.562 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-21T12:13:27.584 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-21T12:13:27.599 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:13:27.600 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:27.600 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:13:27.600 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:27.600 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:13:27.600 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 49 M 2024-05-21T12:13:27.600 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:13:27.600 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 163 k 2024-05-21T12:13:27.600 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T12:13:27.601 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T12:13:27.601 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:27.601 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:13:27.601 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:27.601 INFO:teuthology.orchestra.run.smithi072.stdout:Install 4 Packages 2024-05-21T12:13:27.601 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:27.601 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 50 M 2024-05-21T12:13:27.601 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 211 M 2024-05-21T12:13:27.602 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:13:27.610 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-21T12:13:27.643 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-21T12:13:27.684 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-21T12:13:27.718 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-21T12:13:27.753 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-21T12:13:27.792 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:27.825 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:13:27.864 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-21T12:13:28.006 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-21T12:13:28.061 INFO:teuthology.orchestra.run.smithi072.stdout:(1/4): libcephsqlite-19.0.0-3665.gb0c764b6.el9. 703 kB/s | 163 kB 00:00 2024-05-21T12:13:28.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph 2024-05-21T12:13:28.083 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-21T12:13:28.136 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-21T12:13:28.174 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-21T12:13:28.222 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-21T12:13:28.279 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-21T12:13:28.466 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-21T12:13:28.511 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-21T12:13:28.581 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-21T12:13:28.611 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:42 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:13:28.670 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-21T12:13:28.710 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-21T12:13:28.736 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:13:28.738 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 6.6 k 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 2.4 M 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 1.5 M 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 274 k 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 4.8 M 2024-05-21T12:13:28.739 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 17 M 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T12:13:28.740 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T12:13:28.741 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T12:13:28.742 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:28.743 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:13:28.743 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:28.743 INFO:teuthology.orchestra.run.smithi092.stdout:Install 36 Packages 2024-05-21T12:13:28.743 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:28.743 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 30 M 2024-05-21T12:13:28.744 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 108 M 2024-05-21T12:13:28.744 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:13:28.803 INFO:teuthology.orchestra.run.smithi072.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 313 kB/s | 305 kB 00:00 2024-05-21T12:13:28.836 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-21T12:13:28.837 INFO:teuthology.orchestra.run.smithi072.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 82 kB/s | 64 kB 00:00 2024-05-21T12:13:29.008 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9. 31/36 2024-05-21T12:13:29.038 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/36 2024-05-21T12:13:29.064 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/36 2024-05-21T12:13:29.064 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:29.064 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T12:13:29.064 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T12:13:29.064 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T12:13:29.064 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:29.304 INFO:teuthology.orchestra.run.smithi092.stdout:(1/36): ceph-19.0.0-3665.gb0c764b6.el9.x86_64.r 38 kB/s | 6.6 kB 00:00 2024-05-21T12:13:29.568 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/4 2024-05-21T12:13:29.568 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/4 2024-05-21T12:13:29.569 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/4 2024-05-21T12:13:29.569 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-21T12:13:29.821 INFO:teuthology.orchestra.run.smithi092.stdout:(2/36): ceph-mds-19.0.0-3665.gb0c764b6.el9.x86_ 3.4 MB/s | 2.4 MB 00:00 2024-05-21T12:13:29.859 INFO:teuthology.orchestra.run.smithi092.stdout:(3/36): ceph-mgr-19.0.0-3665.gb0c764b6.el9.x86_ 2.1 MB/s | 1.5 MB 00:00 2024-05-21T12:13:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-21T12:13:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:29.871 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:13:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T12:13:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T12:13:29.872 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:29.872 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:13:29.938 INFO:teuthology.orchestra.run.smithi092.stdout:(4/36): ceph-mon-19.0.0-3665.gb0c764b6.el9.x86_ 7.5 MB/s | 4.8 MB 00:00 2024-05-21T12:13:29.963 INFO:teuthology.orchestra.run.smithi092.stdout:(5/36): ceph-mgr-modules-core-19.0.0-3665.gb0c7 2.5 MB/s | 274 kB 00:00 2024-05-21T12:13:30.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph 2024-05-21T12:13:30.138 INFO:teuthology.orchestra.run.smithi092.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-21T12:13:30.189 INFO:teuthology.orchestra.run.smithi092.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-21T12:13:30.272 INFO:teuthology.orchestra.run.smithi092.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 1.6 MB/s | 135 kB 00:00 2024-05-21T12:13:30.356 INFO:teuthology.orchestra.run.smithi092.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 1.5 MB/s | 125 kB 00:00 2024-05-21T12:13:30.423 INFO:teuthology.orchestra.run.smithi092.stdout:(10/36): python3-cryptography-36.0.1-4.el9.x86_ 2.7 MB/s | 1.2 MB 00:00 2024-05-21T12:13:30.448 INFO:teuthology.orchestra.run.smithi092.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-05-21T12:13:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/36 2024-05-21T12:13:30.507 INFO:teuthology.orchestra.run.smithi092.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-21T12:13:30.532 INFO:teuthology.orchestra.run.smithi092.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-21T12:13:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/36 2024-05-21T12:13:30.532 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:30.533 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T12:13:30.533 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T12:13:30.533 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T12:13:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:30.566 INFO:teuthology.orchestra.run.smithi092.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 592 kB/s | 35 kB 00:00 2024-05-21T12:13:30.591 INFO:teuthology.orchestra.run.smithi092.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-05-21T12:13:30.624 INFO:teuthology.orchestra.run.smithi092.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-05-21T12:13:30.681 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:50 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:13:30.809 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:13:30.811 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:30.811 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:13:30.811 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 6.6 k 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 2.4 M 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 1.5 M 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 274 k 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 4.8 M 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 17 M 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T12:13:30.812 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T12:13:30.813 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T12:13:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T12:13:30.815 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T12:13:30.815 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T12:13:30.815 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:30.815 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:13:30.815 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:30.815 INFO:teuthology.orchestra.run.smithi045.stdout:Install 36 Packages 2024-05-21T12:13:30.815 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:30.815 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 30 M 2024-05-21T12:13:30.816 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 108 M 2024-05-21T12:13:30.816 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:13:30.850 INFO:teuthology.orchestra.run.smithi092.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-05-21T12:13:30.980 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 34/36 2024-05-21T12:13:31.005 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 34/36 2024-05-21T12:13:31.005 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:31.005 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T12:13:31.005 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T12:13:31.005 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T12:13:31.005 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:31.125 INFO:teuthology.orchestra.run.smithi092.stdout:(18/36): ceph-osd-19.0.0-3665.gb0c764b6.el9.x86 13 MB/s | 17 MB 00:01 2024-05-21T12:13:31.233 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 35/36 2024-05-21T12:13:31.258 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 35/36 2024-05-21T12:13:31.258 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:31.258 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T12:13:31.258 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T12:13:31.258 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T12:13:31.258 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:31.322 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 36/36 2024-05-21T12:13:31.371 INFO:teuthology.orchestra.run.smithi092.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 83 kB/s | 43 kB 00:00 2024-05-21T12:13:31.434 INFO:teuthology.orchestra.run.smithi092.stdout:(20/36): luarocks-3.9.2-5.el9.noarch.rpm 187 kB/s | 151 kB 00:00 2024-05-21T12:13:31.610 INFO:teuthology.orchestra.run.smithi092.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 355 kB/s | 172 kB 00:00 2024-05-21T12:13:31.768 INFO:teuthology.orchestra.run.smithi092.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 32 kB/s | 11 kB 00:00 2024-05-21T12:13:31.807 INFO:teuthology.orchestra.run.smithi045.stdout:(1/36): ceph-19.0.0-3665.gb0c764b6.el9.x86_64.r 28 kB/s | 6.6 kB 00:00 2024-05-21T12:13:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 50 kB/s | 18 kB 00:00 2024-05-21T12:13:31.985 INFO:teuthology.orchestra.run.smithi092.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 584 kB/s | 358 kB 00:00 2024-05-21T12:13:32.010 INFO:teuthology.orchestra.run.smithi092.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 96 kB/s | 23 kB 00:00 2024-05-21T12:13:32.015 INFO:teuthology.orchestra.run.smithi045.stdout:(2/36): ceph-mgr-19.0.0-3665.gb0c764b6.el9.x86_ 3.4 MB/s | 1.5 MB 00:00 2024-05-21T12:13:32.061 INFO:teuthology.orchestra.run.smithi092.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-05-21T12:13:32.074 INFO:teuthology.orchestra.run.smithi045.stdout:(3/36): ceph-mds-19.0.0-3665.gb0c764b6.el9.x86_ 4.7 MB/s | 2.4 MB 00:00 2024-05-21T12:13:32.102 INFO:teuthology.orchestra.run.smithi092.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 168 kB/s | 20 kB 00:00 2024-05-21T12:13:32.128 INFO:teuthology.orchestra.run.smithi092.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 395 kB/s | 46 kB 00:00 2024-05-21T12:13:32.132 INFO:teuthology.orchestra.run.smithi045.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3665.gb0c7 4.6 MB/s | 274 kB 00:00 2024-05-21T12:13:32.212 INFO:teuthology.orchestra.run.smithi092.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 523 kB/s | 79 kB 00:00 2024-05-21T12:13:32.392 INFO:teuthology.orchestra.run.smithi045.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 980 kB/s | 253 kB 00:00 2024-05-21T12:13:32.462 INFO:teuthology.orchestra.run.smithi092.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 49 kB/s | 16 kB 00:00 2024-05-21T12:13:32.512 INFO:teuthology.orchestra.run.smithi092.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 665 kB/s | 272 kB 00:00 2024-05-21T12:13:32.537 INFO:teuthology.orchestra.run.smithi092.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 276 kB/s | 90 kB 00:00 2024-05-21T12:13:32.596 INFO:teuthology.orchestra.run.smithi092.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 268 kB/s | 36 kB 00:00 2024-05-21T12:13:32.763 INFO:teuthology.orchestra.run.smithi092.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-05-21T12:13:32.867 INFO:teuthology.orchestra.run.smithi045.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 2.6 MB/s | 1.2 MB 00:00 2024-05-21T12:13:32.904 INFO:teuthology.orchestra.run.smithi092.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 65 kB/s | 20 kB 00:00 2024-05-21T12:13:32.986 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 36/36 2024-05-21T12:13:32.986 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/36 2024-05-21T12:13:32.986 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/36 2024-05-21T12:13:32.986 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/36 2024-05-21T12:13:32.986 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/36 2024-05-21T12:13:32.986 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 5/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9. 6/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-21T12:13:32.987 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 19/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-21T12:13:32.988 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-21T12:13:32.989 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-21T12:13:32.989 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-21T12:13:32.989 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-21T12:13:32.990 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-21T12:13:32.990 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-21T12:13:32.990 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-21T12:13:32.990 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-21T12:13:32.990 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-21T12:13:33.029 INFO:teuthology.orchestra.run.smithi092.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 868 kB/s | 427 kB 00:00 2024-05-21T12:13:33.033 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:33.033 INFO:teuthology.orchestra.run.smithi092.stdout:Total 7.1 MB/s | 30 MB 00:04 2024-05-21T12:13:33.042 INFO:teuthology.orchestra.run.smithi045.stdout:(7/36): ceph-osd-19.0.0-3665.gb0c764b6.el9.x86_ 16 MB/s | 17 MB 00:01 2024-05-21T12:13:33.067 INFO:teuthology.orchestra.run.smithi045.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 532 kB/s | 106 kB 00:00 2024-05-21T12:13:33.134 INFO:teuthology.orchestra.run.smithi045.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 1.8 MB/s | 125 kB 00:00 2024-05-21T12:13:33.175 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:13:33.214 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:13:33.214 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:13:33.251 INFO:teuthology.orchestra.run.smithi045.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.8 MB/s | 215 kB 00:00 2024-05-21T12:13:33.276 INFO:teuthology.orchestra.run.smithi045.stdout:(11/36): python3-pycparser-2.20-6.el9.noarch.rp 578 kB/s | 135 kB 00:00 2024-05-21T12:13:33.296 INFO:teuthology.orchestra.run.smithi072.stdout:(4/4): ceph-test-19.0.0-3665.gb0c764b6.el9.x86_ 9.0 MB/s | 49 MB 00:05 2024-05-21T12:13:33.298 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:33.298 INFO:teuthology.orchestra.run.smithi072.stdout:Total 8.7 MB/s | 50 MB 00:05 2024-05-21T12:13:33.326 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:13:33.343 INFO:teuthology.orchestra.run.smithi045.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-21T12:13:33.347 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:13:33.347 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:13:33.402 INFO:teuthology.orchestra.run.smithi045.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 597 kB/s | 35 kB 00:00 2024-05-21T12:13:33.419 INFO:teuthology.orchestra.run.smithi045.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-05-21T12:13:33.461 INFO:teuthology.orchestra.run.smithi045.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-05-21T12:13:33.486 INFO:teuthology.orchestra.run.smithi045.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-05-21T12:13:33.519 INFO:teuthology.orchestra.run.smithi045.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 382 kB/s | 22 kB 00:00 2024-05-21T12:13:33.642 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:13:33.642 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:13:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 208 kB/s | 43 kB 00:00 2024-05-21T12:13:33.731 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:13:33.731 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:13:33.796 INFO:teuthology.orchestra.run.smithi045.stdout:(19/36): ceph-mon-19.0.0-3665.gb0c764b6.el9.x86 2.4 MB/s | 4.8 MB 00:01 2024-05-21T12:13:33.821 INFO:teuthology.orchestra.run.smithi045.stdout:(20/36): luarocks-3.9.2-5.el9.noarch.rpm 453 kB/s | 151 kB 00:00 2024-05-21T12:13:33.904 INFO:teuthology.orchestra.run.smithi045.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-05-21T12:13:33.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T12:13:33.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T12:13:33.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T12:13:33.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T12:13:33.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T12:13:33.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T12:13:33.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T12:13:33.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T12:13:33.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T12:13:33.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T12:13:33.927 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:33.928 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:13:33.980 INFO:teuthology.orchestra.run.smithi045.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 236 kB/s | 18 kB 00:00 2024-05-21T12:13:34.039 INFO:teuthology.orchestra.run.smithi045.stdout:(23/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 555 kB/s | 172 kB 00:00 2024-05-21T12:13:34.063 INFO:teuthology.orchestra.run.smithi045.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 277 kB/s | 23 kB 00:00 2024-05-21T12:13:34.114 INFO:teuthology.orchestra.run.smithi045.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 259 kB/s | 19 kB 00:00 2024-05-21T12:13:34.139 INFO:teuthology.orchestra.run.smithi045.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-05-21T12:13:34.164 INFO:teuthology.orchestra.run.smithi045.stdout:(27/36): python3-cherrypy-18.6.1-2.el9.noarch.r 973 kB/s | 358 kB 00:00 2024-05-21T12:13:34.248 INFO:teuthology.orchestra.run.smithi045.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 346 kB/s | 46 kB 00:00 2024-05-21T12:13:34.270 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-base 2024-05-21T12:13:34.276 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:13:34.348 INFO:teuthology.orchestra.run.smithi045.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 164 kB/s | 16 kB 00:00 2024-05-21T12:13:34.358 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-21T12:13:34.407 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-21T12:13:34.407 INFO:teuthology.orchestra.run.smithi045.stdout:(30/36): python3-more-itertools-8.12.0-2.el9.no 295 kB/s | 79 kB 00:00 2024-05-21T12:13:34.434 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/4 2024-05-21T12:13:34.457 INFO:teuthology.orchestra.run.smithi045.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 827 kB/s | 90 kB 00:00 2024-05-21T12:13:34.549 INFO:teuthology.orchestra.run.smithi045.stdout:(32/36): python3-pecan-1.4.2-3.el9.noarch.rpm 708 kB/s | 272 kB 00:00 2024-05-21T12:13:34.575 INFO:teuthology.orchestra.run.smithi045.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 214 kB/s | 36 kB 00:00 2024-05-21T12:13:34.599 INFO:teuthology.orchestra.run.smithi045.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-05-21T12:13:34.658 INFO:teuthology.orchestra.run.smithi045.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 239 kB/s | 20 kB 00:00 2024-05-21T12:13:34.683 INFO:teuthology.orchestra.run.smithi045.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-05-21T12:13:34.686 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:34.686 INFO:teuthology.orchestra.run.smithi045.stdout:Total 7.8 MB/s | 30 MB 00:03 2024-05-21T12:13:34.811 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:59 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:13:34.834 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:13:34.877 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:13:34.878 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:13:34.890 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:13:34.891 INFO:teuthology.orchestra.run.smithi137.stdout:Package ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:13:34.933 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:13:34.934 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:13:34.934 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:13:34.957 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-21T12:13:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-21T12:13:35.013 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install cephadm 2024-05-21T12:13:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-21T12:13:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-21T12:13:35.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/36 2024-05-21T12:13:35.177 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-21T12:13:35.269 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-21T12:13:35.346 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:13:35.346 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:13:35.347 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-21T12:13:35.396 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-21T12:13:35.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-21T12:13:35.478 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-21T12:13:35.512 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-21T12:13:35.542 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:00 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:13:35.545 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-21T12:13:35.586 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-21T12:13:35.621 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-21T12:13:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-21T12:13:35.665 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:13:35.666 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:35.666 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:13:35.666 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:35.666 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 764 k 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 764 k 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 770 k 2024-05-21T12:13:35.667 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:13:35.686 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-21T12:13:35.759 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-21T12:13:35.897 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-21T12:13:35.965 INFO:teuthology.orchestra.run.smithi137.stdout:cephadm-19.0.0-3665.gb0c764b6.el9.noarch.rpm 2.5 MB/s | 764 kB 00:00 2024-05-21T12:13:35.965 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:35.966 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.5 MB/s | 764 kB 00:00 2024-05-21T12:13:35.966 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:13:35.970 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:13:35.971 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:13:35.977 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:13:35.977 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:13:35.985 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-21T12:13:36.038 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-21T12:13:36.049 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:13:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-21T12:13:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-21T12:13:36.190 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-21T12:13:36.376 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-21T12:13:36.422 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-21T12:13:36.483 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:36.509 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-21T12:13:36.568 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:36.598 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-21T12:13:36.646 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-21T12:13:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:13:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-21T12:13:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-21T12:13:36.873 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-21T12:13:36.906 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-21T12:13:36.930 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:36.935 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9. 31/36 2024-05-21T12:13:36.965 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/36 2024-05-21T12:13:36.982 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-21T12:13:36.992 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/36 2024-05-21T12:13:36.992 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:36.992 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T12:13:36.992 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T12:13:36.992 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T12:13:36.992 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:37.147 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/36 2024-05-21T12:13:37.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:37.165 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:37.165 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:13:37.165 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:37.165 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:37.165 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:13:37.179 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-21T12:13:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-21T12:13:37.360 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-21T12:13:37.375 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-immutable-object-cache 2024-05-21T12:13:37.407 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-21T12:13:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-21T12:13:37.471 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-21T12:13:37.506 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-21T12:13:37.539 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-21T12:13:37.580 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-21T12:13:37.623 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-21T12:13:37.657 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-21T12:13:37.696 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-21T12:13:37.773 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-21T12:13:37.914 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:02 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:13:37.926 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-21T12:13:38.024 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-21T12:13:38.037 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 145 k 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-05-21T12:13:38.039 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:38.040 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 145 k 2024-05-21T12:13:38.040 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 435 k 2024-05-21T12:13:38.040 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:13:38.192 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-21T12:13:38.229 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-21T12:13:38.246 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-immutable-object-cache-19.0.0-3665.gb0c764 701 kB/s | 145 kB 00:00 2024-05-21T12:13:38.247 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:38.247 INFO:teuthology.orchestra.run.smithi137.stdout:Total 694 kB/s | 145 kB 00:00 2024-05-21T12:13:38.247 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:13:38.255 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:13:38.255 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:13:38.288 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-21T12:13:38.311 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:13:38.312 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:13:38.382 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-21T12:13:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/36 2024-05-21T12:13:38.452 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/36 2024-05-21T12:13:38.452 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:38.452 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T12:13:38.452 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T12:13:38.453 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T12:13:38.453 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:38.460 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:13:38.539 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:38.564 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:38.564 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:38.564 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T12:13:38.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:38.590 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-21T12:13:38.677 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-21T12:13:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-21T12:13:38.867 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-21T12:13:38.892 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 34/36 2024-05-21T12:13:38.907 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-21T12:13:38.916 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 34/36 2024-05-21T12:13:38.916 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:38.916 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T12:13:38.916 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T12:13:38.916 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T12:13:38.916 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:39.041 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-21T12:13:39.087 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/4 2024-05-21T12:13:39.134 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 35/36 2024-05-21T12:13:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 35/36 2024-05-21T12:13:39.158 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:39.158 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T12:13:39.158 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T12:13:39.158 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T12:13:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:39.172 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/4 2024-05-21T12:13:39.214 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9. 31/36 2024-05-21T12:13:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 36/36 2024-05-21T12:13:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/36 2024-05-21T12:13:39.270 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/36 2024-05-21T12:13:39.270 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:39.270 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T12:13:39.270 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T12:13:39.270 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T12:13:39.270 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:39.447 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:39.447 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:39.447 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:13:39.447 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:39.447 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:39.447 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:13:39.682 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr 2024-05-21T12:13:40.224 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:05 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:13:40.307 INFO:teuthology.orchestra.run.smithi137.stdout:Package ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:13:40.349 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:13:40.350 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:13:40.350 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:13:40.421 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-dashboard 2024-05-21T12:13:40.784 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/36 2024-05-21T12:13:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/36 2024-05-21T12:13:40.814 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:40.814 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T12:13:40.814 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T12:13:40.814 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T12:13:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:40.951 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:05 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 36/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 5/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9. 6/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-21T12:13:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 19/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-21T12:13:41.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-21T12:13:41.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-21T12:13:41.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-21T12:13:41.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-21T12:13:41.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-21T12:13:41.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-21T12:13:41.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-21T12:13:41.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-21T12:13:41.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-21T12:13:41.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-21T12:13:41.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-21T12:13:41.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-21T12:13:41.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-21T12:13:41.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-21T12:13:41.073 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:13:41.074 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 3.5 M 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 34 k 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 16 k 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T12:13:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T12:13:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T12:13:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T12:13:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T12:13:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:41.076 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:13:41.076 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:13:41.076 INFO:teuthology.orchestra.run.smithi137.stdout:Install 12 Packages 2024-05-21T12:13:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:41.077 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 8.6 M 2024-05-21T12:13:41.077 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 104 M 2024-05-21T12:13:41.077 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:13:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 34/36 2024-05-21T12:13:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 34/36 2024-05-21T12:13:41.303 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:41.303 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T12:13:41.303 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T12:13:41.303 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T12:13:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 35/36 2024-05-21T12:13:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 35/36 2024-05-21T12:13:41.562 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:41.562 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T12:13:41.562 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T12:13:41.562 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T12:13:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 36/36 2024-05-21T12:13:41.817 INFO:teuthology.orchestra.run.smithi137.stdout:(1/12): ceph-grafana-dashboards-19.0.0-3665.gb0 173 kB/s | 34 kB 00:00 2024-05-21T12:13:41.930 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-21T12:13:41.930 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:41.930 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:13:41.930 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:41.930 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:41.930 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T12:13:41.931 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T12:13:41.932 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T12:13:41.933 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:41.934 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:13:41.950 INFO:teuthology.orchestra.run.smithi137.stdout:(2/12): ceph-prometheus-alerts-19.0.0-3665.gb0c 50 kB/s | 16 kB 00:00 2024-05-21T12:13:42.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-base 2024-05-21T12:13:42.726 INFO:teuthology.orchestra.run.smithi137.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 347 kB/s | 269 kB 00:00 2024-05-21T12:13:42.768 INFO:teuthology.orchestra.run.smithi137.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-05-21T12:13:42.905 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:56 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:13:42.985 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:13:43.007 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/4 2024-05-21T12:13:43.007 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/4 2024-05-21T12:13:43.007 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/4 2024-05-21T12:13:43.007 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-21T12:13:43.026 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:13:43.027 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:13:43.027 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:13:43.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install cephadm 2024-05-21T12:13:43.251 INFO:teuthology.orchestra.run.smithi137.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.6 MB/s | 863 kB 00:00 2024-05-21T12:13:43.276 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 36/36 2024-05-21T12:13:43.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/36 2024-05-21T12:13:43.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/36 2024-05-21T12:13:43.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/36 2024-05-21T12:13:43.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/36 2024-05-21T12:13:43.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 5/36 2024-05-21T12:13:43.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9. 6/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-21T12:13:43.277 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 19/36 2024-05-21T12:13:43.278 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-21T12:13:43.278 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-21T12:13:43.278 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-21T12:13:43.278 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-21T12:13:43.278 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-21T12:13:43.278 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-21T12:13:43.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-21T12:13:43.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-21T12:13:43.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-21T12:13:43.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-21T12:13:43.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-21T12:13:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-21T12:13:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-21T12:13:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-21T12:13:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-21T12:13:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-21T12:13:43.309 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-21T12:13:43.309 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:43.309 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:13:43.309 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:43.310 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:43.310 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T12:13:43.310 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T12:13:43.310 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:43.310 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:13:43.443 INFO:teuthology.orchestra.run.smithi137.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 812 kB/s | 548 kB 00:00 2024-05-21T12:13:43.568 INFO:teuthology.orchestra.run.smithi137.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 61 kB/s | 19 kB 00:00 2024-05-21T12:13:43.619 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph 2024-05-21T12:13:43.621 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:57 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:13:43.742 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:13:43.743 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:43.743 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:13:43.743 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 764 k 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 764 k 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 770 k 2024-05-21T12:13:43.744 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:13:43.777 INFO:teuthology.orchestra.run.smithi137.stdout:(8/12): ceph-mgr-dashboard-19.0.0-3665.gb0c764b 1.6 MB/s | 3.5 MB 00:02 2024-05-21T12:13:43.886 INFO:teuthology.orchestra.run.smithi137.stdout:(9/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 4.6 MB/s | 2.0 MB 00:00 2024-05-21T12:13:43.911 INFO:teuthology.orchestra.run.smithi137.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 422 kB/s | 144 kB 00:00 2024-05-21T12:13:44.028 INFO:teuthology.orchestra.run.smithi137.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.3 MB/s | 188 kB 00:00 2024-05-21T12:13:44.044 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm-19.0.0-3665.gb0c764b6.el9.noarch.rpm 2.5 MB/s | 764 kB 00:00 2024-05-21T12:13:44.044 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:44.044 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.5 MB/s | 764 kB 00:00 2024-05-21T12:13:44.045 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:13:44.050 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:13:44.050 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:13:44.055 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:13:44.056 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:13:44.103 INFO:teuthology.orchestra.run.smithi137.stdout:(12/12): python3-repoze-lru-0.7-16.el9.noarch.r 94 kB/s | 31 kB 00:00 2024-05-21T12:13:44.104 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:44.104 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.9 MB/s | 8.6 MB 00:03 2024-05-21T12:13:44.129 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:13:44.156 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:11 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:13:44.157 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:13:44.176 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:13:44.176 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:13:44.225 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-21T12:13:44.225 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:44.225 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:13:44.225 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:44.225 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:44.225 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:44.225 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T12:13:44.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T12:13:44.227 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T12:13:44.228 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T12:13:44.228 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T12:13:44.228 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T12:13:44.228 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T12:13:44.228 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T12:13:44.228 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T12:13:44.228 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:44.228 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:13:44.288 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:13:44.290 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:44.290 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:13:44.290 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:44.290 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:13:44.290 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 6.6 k 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 2.4 M 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 1.5 M 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 274 k 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 4.8 M 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 17 M 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T12:13:44.291 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T12:13:44.292 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T12:13:44.293 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T12:13:44.293 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T12:13:44.293 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T12:13:44.293 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T12:13:44.293 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T12:13:44.293 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T12:13:44.293 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T12:13:44.293 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout:Install 36 Packages 2024-05-21T12:13:44.294 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:44.295 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 30 M 2024-05-21T12:13:44.295 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 108 M 2024-05-21T12:13:44.295 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:13:44.355 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:13:44.356 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:13:44.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-base 2024-05-21T12:13:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:44.979 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:13:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:45.053 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-21T12:13:45.101 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-21T12:13:45.124 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:05 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:13:45.152 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-21T12:13:45.191 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-21T12:13:45.207 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:13:45.254 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:13:45.255 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:13:45.255 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:13:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:45.280 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:13:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:45.281 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:13:45.305 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-21T12:13:45.306 INFO:teuthology.orchestra.run.smithi072.stdout:(1/36): ceph-19.0.0-3665.gb0c764b6.el9.x86_64.r 35 kB/s | 6.6 kB 00:00 2024-05-21T12:13:45.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install cephadm 2024-05-21T12:13:45.441 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-21T12:13:45.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-immutable-object-cache 2024-05-21T12:13:45.540 INFO:teuthology.orchestra.run.smithi072.stdout:(2/36): ceph-mgr-19.0.0-3665.gb0c764b6.el9.x86_ 3.6 MB/s | 1.5 MB 00:00 2024-05-21T12:13:45.544 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-21T12:13:45.587 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-21T12:13:45.608 INFO:teuthology.orchestra.run.smithi072.stdout:(3/36): ceph-mds-19.0.0-3665.gb0c764b6.el9.x86_ 4.9 MB/s | 2.4 MB 00:00 2024-05-21T12:13:45.616 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-21T12:13:45.648 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9 10/12 2024-05-21T12:13:45.675 INFO:teuthology.orchestra.run.smithi072.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3665.gb0c7 4.0 MB/s | 274 kB 00:00 2024-05-21T12:13:45.884 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:05 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:13:45.909 INFO:teuthology.orchestra.run.smithi072.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-21T12:13:46.012 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 764 k 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-05-21T12:13:46.013 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:46.014 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 764 k 2024-05-21T12:13:46.014 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 770 k 2024-05-21T12:13:46.014 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:13:46.028 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:00 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:13:46.153 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 145 k 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-21T12:13:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:46.155 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 145 k 2024-05-21T12:13:46.155 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 435 k 2024-05-21T12:13:46.155 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:13:46.534 INFO:teuthology.orchestra.run.smithi072.stdout:(6/36): ceph-osd-19.0.0-3665.gb0c764b6.el9.x86_ 17 MB/s | 17 MB 00:00 2024-05-21T12:13:46.585 INFO:teuthology.orchestra.run.smithi072.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 1.8 MB/s | 1.2 MB 00:00 2024-05-21T12:13:46.651 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-immutable-object-cache-19.0.0-3665.gb0c764 292 kB/s | 145 kB 00:00 2024-05-21T12:13:46.651 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:46.651 INFO:teuthology.orchestra.run.smithi092.stdout:Total 291 kB/s | 145 kB 00:00 2024-05-21T12:13:46.652 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:13:46.659 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:13:46.659 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:13:46.685 INFO:teuthology.orchestra.run.smithi072.stdout:(8/36): ceph-mon-19.0.0-3665.gb0c764b6.el9.x86_ 3.4 MB/s | 4.8 MB 00:01 2024-05-21T12:13:46.693 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el 11/12 2024-05-21T12:13:46.716 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:13:46.716 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:13:46.719 INFO:teuthology.orchestra.run.smithi072.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 1.0 MB/s | 135 kB 00:00 2024-05-21T12:13:46.723 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 12/12 2024-05-21T12:13:46.778 INFO:teuthology.orchestra.run.smithi072.stdout:(10/36): python3-ply-3.11-14.el9.noarch.rpm 438 kB/s | 106 kB 00:00 2024-05-21T12:13:46.837 INFO:teuthology.orchestra.run.smithi072.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.8 MB/s | 215 kB 00:00 2024-05-21T12:13:46.844 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm-19.0.0-3665.gb0c764b6.el9.noarch.rpm 920 kB/s | 764 kB 00:00 2024-05-21T12:13:46.844 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:46.845 INFO:teuthology.orchestra.run.smithi045.stdout:Total 918 kB/s | 764 kB 00:00 2024-05-21T12:13:46.845 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:13:46.851 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:13:46.851 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:13:46.857 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:13:46.858 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:13:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:13:46.889 INFO:teuthology.orchestra.run.smithi072.stdout:(12/36): python3-requests-2.25.1-8.el9.noarch.r 615 kB/s | 125 kB 00:00 2024-05-21T12:13:46.923 INFO:teuthology.orchestra.run.smithi072.stdout:(13/36): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-21T12:13:46.930 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:13:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:46.948 INFO:teuthology.orchestra.run.smithi072.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-05-21T12:13:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:46.970 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:46.970 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T12:13:46.970 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:46.973 INFO:teuthology.orchestra.run.smithi072.stdout:(15/36): python3-markupsafe-1.1.1-12.el9.x86_64 410 kB/s | 35 kB 00:00 2024-05-21T12:13:46.999 INFO:teuthology.orchestra.run.smithi072.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 669 kB/s | 51 kB 00:00 2024-05-21T12:13:47.025 INFO:teuthology.orchestra.run.smithi072.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 547 kB/s | 42 kB 00:00 2024-05-21T12:13:47.159 INFO:teuthology.orchestra.run.smithi072.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-05-21T12:13:47.301 INFO:teuthology.orchestra.run.smithi072.stdout:(19/36): luarocks-3.9.2-5.el9.noarch.rpm 503 kB/s | 151 kB 00:00 2024-05-21T12:13:47.326 INFO:teuthology.orchestra.run.smithi072.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 144 kB/s | 43 kB 00:00 2024-05-21T12:13:47.369 INFO:teuthology.orchestra.run.smithi072.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 255 kB/s | 11 kB 00:00 2024-05-21T12:13:47.391 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:47.419 INFO:teuthology.orchestra.run.smithi072.stdout:(22/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 663 kB/s | 172 kB 00:00 2024-05-21T12:13:47.445 INFO:teuthology.orchestra.run.smithi072.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 233 kB/s | 18 kB 00:00 2024-05-21T12:13:47.470 INFO:teuthology.orchestra.run.smithi072.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 457 kB/s | 23 kB 00:00 2024-05-21T12:13:47.504 INFO:teuthology.orchestra.run.smithi072.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-05-21T12:13:47.529 INFO:teuthology.orchestra.run.smithi072.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 230 kB/s | 19 kB 00:00 2024-05-21T12:13:47.555 INFO:teuthology.orchestra.run.smithi072.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 233 kB/s | 20 kB 00:00 2024-05-21T12:13:47.582 INFO:teuthology.orchestra.run.smithi072.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 598 kB/s | 46 kB 00:00 2024-05-21T12:13:47.609 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:47.649 INFO:teuthology.orchestra.run.smithi072.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 245 kB/s | 16 kB 00:00 2024-05-21T12:13:47.675 INFO:teuthology.orchestra.run.smithi072.stdout:(30/36): python3-more-itertools-8.12.0-2.el9.no 660 kB/s | 79 kB 00:00 2024-05-21T12:13:47.742 INFO:teuthology.orchestra.run.smithi072.stdout:(31/36): python3-tempora-5.0.0-2.el9.noarch.rpm 535 kB/s | 36 kB 00:00 2024-05-21T12:13:47.767 INFO:teuthology.orchestra.run.smithi072.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 763 kB/s | 90 kB 00:00 2024-05-21T12:13:47.793 INFO:teuthology.orchestra.run.smithi072.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-05-21T12:13:47.852 INFO:teuthology.orchestra.run.smithi072.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 340 kB/s | 20 kB 00:00 2024-05-21T12:13:47.854 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:47.855 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:13:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:47.855 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:47.855 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:13:47.919 INFO:teuthology.orchestra.run.smithi072.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-05-21T12:13:47.966 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 12/12 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el 1/12 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 2/12 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9 3/12 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-21T12:13:48.002 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-21T12:13:48.003 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-21T12:13:48.003 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-21T12:13:48.003 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-21T12:13:48.019 INFO:teuthology.orchestra.run.smithi072.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.7 MB/s | 427 kB 00:00 2024-05-21T12:13:48.023 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:48.024 INFO:teuthology.orchestra.run.smithi072.stdout:Total 8.1 MB/s | 30 MB 00:03 2024-05-21T12:13:48.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr 2024-05-21T12:13:48.166 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:13:48.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:13:48.203 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:48.203 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:13:48.203 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:48.204 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:48.204 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:13:48.205 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:13:48.205 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:13:48.306 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-21T12:13:48.306 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:13:48.306 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T12:13:48.306 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:48.306 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:48.306 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:48.307 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:13:48.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-immutable-object-cache 2024-05-21T12:13:48.554 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-21T12:13:48.629 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:02 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:13:48.673 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:13:48.674 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:13:48.712 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:13:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:13:48.755 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:13:48.755 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:13:48.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-dashboard 2024-05-21T12:13:48.988 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:08 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:13:49.086 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:14 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:13:49.116 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:13:49.117 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:49.117 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-05-21T12:13:49.117 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:49.117 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:13:49.117 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 145 k 2024-05-21T12:13:49.118 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:49.118 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:13:49.118 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:49.118 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-05-21T12:13:49.118 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:49.118 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 145 k 2024-05-21T12:13:49.118 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 435 k 2024-05-21T12:13:49.118 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:13:49.207 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:13:49.208 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-05-21T12:13:49.208 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:13:49.208 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 7.4 M 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T12:13:49.209 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout:Install 12 Packages 2024-05-21T12:13:49.210 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:13:49.211 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 38 M 2024-05-21T12:13:49.211 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 209 M 2024-05-21T12:13:49.211 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:13:49.334 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-immutable-object-cache-19.0.0-3665.gb0c764 671 kB/s | 145 kB 00:00 2024-05-21T12:13:49.335 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:49.335 INFO:teuthology.orchestra.run.smithi045.stdout:Total 665 kB/s | 145 kB 00:00 2024-05-21T12:13:49.335 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:13:49.343 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:13:49.344 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:13:49.357 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:03 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:13:49.403 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:13:49.403 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:13:49.479 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:13:49.480 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:49.480 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:13:49.480 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:49.481 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:13:49.481 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 3.5 M 2024-05-21T12:13:49.481 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:13:49.481 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T12:13:49.481 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 34 k 2024-05-21T12:13:49.481 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 16 k 2024-05-21T12:13:49.482 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T12:13:49.482 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T12:13:49.482 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T12:13:49.482 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T12:13:49.482 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T12:13:49.482 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T12:13:49.483 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T12:13:49.483 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T12:13:49.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:49.483 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:13:49.483 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:13:49.483 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-05-21T12:13:49.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:49.484 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 8.6 M 2024-05-21T12:13:49.484 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 104 M 2024-05-21T12:13:49.484 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:13:49.554 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:13:49.611 INFO:teuthology.orchestra.run.smithi137.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 653 kB/s | 195 kB 00:00 2024-05-21T12:13:49.635 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:49.662 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:49.662 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:49.662 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T12:13:49.662 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:49.828 INFO:teuthology.orchestra.run.smithi137.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-05-21T12:13:49.938 INFO:teuthology.orchestra.run.smithi137.stdout:(3/12): ceph-mgr-diskprediction-local-19.0.0-36 12 MB/s | 7.4 MB 00:00 2024-05-21T12:13:49.944 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:13:50.006 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-21T12:13:50.062 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-21T12:13:50.101 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-21T12:13:50.181 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-21T12:13:50.236 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/36 2024-05-21T12:13:50.269 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-21T12:13:50.280 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): ceph-grafana-dashboards-19.0.0-3665.gb0 94 kB/s | 34 kB 00:00 2024-05-21T12:13:50.305 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): ceph-prometheus-alerts-19.0.0-3665.gb0c 42 kB/s | 16 kB 00:00 2024-05-21T12:13:50.360 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-21T12:13:50.364 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): ceph-mgr-dashboard-19.0.0-3665.gb0c764b 7.9 MB/s | 3.5 MB 00:00 2024-05-21T12:13:50.445 INFO:teuthology.orchestra.run.smithi137.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 39 kB/s | 33 kB 00:00 2024-05-21T12:13:50.448 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-21T12:13:50.493 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-21T12:13:50.536 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-21T12:13:50.542 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:13:50.542 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:50.543 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:13:50.543 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:50.543 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:50.543 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:13:50.567 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-21T12:13:50.602 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-21T12:13:50.636 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-21T12:13:50.639 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 807 kB/s | 269 kB 00:00 2024-05-21T12:13:50.654 INFO:teuthology.orchestra.run.smithi137.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 26 kB/s | 18 kB 00:00 2024-05-21T12:13:50.677 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-21T12:13:50.729 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-21T12:13:50.747 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr 2024-05-21T12:13:50.758 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-21T12:13:50.796 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-21T12:13:50.856 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.7 MB/s | 863 kB 00:00 2024-05-21T12:13:50.869 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-21T12:13:50.890 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.7 MB/s | 1.0 MB 00:00 2024-05-21T12:13:51.016 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-21T12:13:51.032 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.4 MB/s | 548 kB 00:00 2024-05-21T12:13:51.049 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): grpc-data-1.46.7-10.el9.noarch.rpm 101 kB/s | 19 kB 00:00 2024-05-21T12:13:51.100 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-21T12:13:51.155 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-21T12:13:51.182 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 230 kB/s | 31 kB 00:00 2024-05-21T12:13:51.194 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-21T12:13:51.243 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-21T12:13:51.262 INFO:teuthology.orchestra.run.smithi137.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 43 kB/s | 35 kB 00:00 2024-05-21T12:13:51.304 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:11 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:13:51.308 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-21T12:13:51.316 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 509 kB/s | 144 kB 00:00 2024-05-21T12:13:51.384 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 942 kB/s | 188 kB 00:00 2024-05-21T12:13:51.389 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:13:51.435 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:13:51.436 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:13:51.436 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:13:51.437 INFO:teuthology.orchestra.run.smithi137.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 1.9 MB/s | 3.0 MB 00:01 2024-05-21T12:13:51.495 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-21T12:13:51.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-dashboard 2024-05-21T12:13:51.545 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-21T12:13:51.623 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-21T12:13:51.722 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-21T12:13:51.749 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.4 MB/s | 2.0 MB 00:00 2024-05-21T12:13:51.751 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:51.751 INFO:teuthology.orchestra.run.smithi092.stdout:Total 3.8 MB/s | 8.6 MB 00:02 2024-05-21T12:13:51.763 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-21T12:13:51.805 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:13:51.824 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:13:51.824 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:13:51.889 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-21T12:13:52.002 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:13:52.002 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:13:52.065 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9. 31/36 2024-05-21T12:13:52.066 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:12 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:13:52.094 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/36 2024-05-21T12:13:52.120 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 32/36 2024-05-21T12:13:52.120 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:52.120 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T12:13:52.120 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T12:13:52.120 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T12:13:52.121 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:52.195 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 3.5 M 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 34 k 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 16 k 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T12:13:52.196 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-05-21T12:13:52.197 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:52.198 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 8.6 M 2024-05-21T12:13:52.198 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 104 M 2024-05-21T12:13:52.198 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:13:52.230 INFO:teuthology.orchestra.run.smithi137.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.0 MB/s | 4.7 MB 00:01 2024-05-21T12:13:52.255 INFO:teuthology.orchestra.run.smithi137.stdout:(9/12): python3-devel-3.9.18-3.el9.x86_64.rpm 246 kB/s | 244 kB 00:00 2024-05-21T12:13:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:13:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-21T12:13:52.744 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-21T12:13:52.791 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-21T12:13:52.830 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-21T12:13:52.855 INFO:teuthology.orchestra.run.smithi137.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-05-21T12:13:52.979 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-21T12:13:53.097 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): ceph-grafana-dashboards-19.0.0-3665.gb0 172 kB/s | 34 kB 00:00 2024-05-21T12:13:53.106 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-21T12:13:53.201 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-21T12:13:53.213 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): ceph-prometheus-alerts-19.0.0-3665.gb0c 52 kB/s | 16 kB 00:00 2024-05-21T12:13:53.230 INFO:teuthology.orchestra.run.smithi137.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 251 kB/s | 250 kB 00:00 2024-05-21T12:13:53.243 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-21T12:13:53.271 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-21T12:13:53.304 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9 10/12 2024-05-21T12:13:53.563 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/36 2024-05-21T12:13:53.590 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 33/36 2024-05-21T12:13:53.590 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:53.590 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T12:13:53.591 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T12:13:53.591 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T12:13:53.591 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:54.039 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 34/36 2024-05-21T12:13:54.047 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): ceph-mgr-dashboard-19.0.0-3665.gb0c764b 3.1 MB/s | 3.5 MB 00:01 2024-05-21T12:13:54.063 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 34/36 2024-05-21T12:13:54.063 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:54.063 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T12:13:54.063 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T12:13:54.063 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T12:13:54.063 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:54.197 INFO:teuthology.orchestra.run.smithi137.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.4 MB/s | 16 MB 00:01 2024-05-21T12:13:54.198 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:54.198 INFO:teuthology.orchestra.run.smithi137.stdout:Total 7.6 MB/s | 38 MB 00:04 2024-05-21T12:13:54.256 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 258 kB/s | 269 kB 00:01 2024-05-21T12:13:54.292 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 35/36 2024-05-21T12:13:54.316 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 35/36 2024-05-21T12:13:54.316 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:13:54.316 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T12:13:54.316 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T12:13:54.316 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T12:13:54.316 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:54.348 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 826 kB/s | 1.0 MB 00:01 2024-05-21T12:13:54.353 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el 11/12 2024-05-21T12:13:54.381 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 36/36 2024-05-21T12:13:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 12/12 2024-05-21T12:13:54.473 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-05-21T12:13:54.554 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:13:54.581 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:13:54.581 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:13:54.590 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.6 MB/s | 863 kB 00:00 2024-05-21T12:13:54.615 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.5 MB/s | 548 kB 00:00 2024-05-21T12:13:54.657 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 739 kB/s | 31 kB 00:00 2024-05-21T12:13:54.732 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.5 MB/s | 188 kB 00:00 2024-05-21T12:13:54.816 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 639 kB/s | 144 kB 00:00 2024-05-21T12:13:54.858 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.3 MB/s | 2.0 MB 00:00 2024-05-21T12:13:54.860 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:13:54.860 INFO:teuthology.orchestra.run.smithi045.stdout:Total 3.2 MB/s | 8.6 MB 00:02 2024-05-21T12:13:54.911 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:13:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:13:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:13:55.002 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:13:55.002 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:13:55.131 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:13:55.131 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:13:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 12/12 2024-05-21T12:13:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el 1/12 2024-05-21T12:13:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 2/12 2024-05-21T12:13:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9 3/12 2024-05-21T12:13:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-21T12:13:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-21T12:13:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-21T12:13:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-21T12:13:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-21T12:13:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-21T12:13:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-21T12:13:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-21T12:13:55.766 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:13:55.837 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-21T12:13:55.849 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:13:55.879 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-21T12:13:55.926 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-21T12:13:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T12:13:55.970 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T12:13:55.971 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T12:13:55.971 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T12:13:55.971 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T12:13:55.971 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T12:13:55.971 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:55.971 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:13:55.983 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-21T12:13:56.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-21T12:13:56.021 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-21T12:13:56.038 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-21T12:13:56.143 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-21T12:13:56.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-21T12:13:56.198 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-21T12:13:56.229 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-21T12:13:56.288 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-21T12:13:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-21T12:13:56.380 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-21T12:13:56.428 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-21T12:13:56.454 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-21T12:13:56.478 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9 10/12 2024-05-21T12:13:56.746 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:10 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:13:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:13:56.878 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-21T12:13:56.878 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:13:56.878 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 7.4 M 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T12:13:56.879 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T12:13:56.880 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T12:13:56.880 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T12:13:56.880 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:56.880 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:13:56.880 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-21T12:13:56.880 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-05-21T12:13:56.880 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:13:56.881 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 38 M 2024-05-21T12:13:56.881 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 209 M 2024-05-21T12:13:56.881 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:13:56.963 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 36/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 3/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 4/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 5/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9. 6/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-21T12:13:56.964 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 19/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-21T12:13:56.965 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-21T12:13:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-21T12:13:56.966 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-21T12:13:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-21T12:13:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-21T12:13:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-21T12:13:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-21T12:13:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-21T12:13:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-21T12:13:56.967 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-21T12:13:56.973 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-21T12:13:57.076 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-21T12:13:57.590 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el 11/12 2024-05-21T12:13:57.628 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 12/12 2024-05-21T12:13:57.941 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-21T12:13:57.941 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:57.941 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:13:57.941 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:57.941 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:57.941 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:57.941 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T12:13:57.942 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T12:13:57.946 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:57.947 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:13:58.167 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): ceph-mgr-diskprediction-local-19.0.0-36 7.5 MB/s | 7.4 MB 00:00 2024-05-21T12:13:58.192 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 193 kB/s | 195 kB 00:01 2024-05-21T12:13:58.234 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 760 kB/s | 803 kB 00:01 2024-05-21T12:13:58.343 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-21T12:13:58.359 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-base 2024-05-21T12:13:58.810 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 32 kB/s | 18 kB 00:00 2024-05-21T12:13:58.835 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): flexiblas-3.0.4-8.el9.x86_64.rpm 49 kB/s | 33 kB 00:00 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 12/12 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el 1/12 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 2/12 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9 3/12 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-21T12:13:58.875 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-21T12:13:58.876 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-21T12:13:58.876 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-21T12:13:58.876 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-21T12:13:58.891 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:25 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:13:58.971 INFO:teuthology.orchestra.run.smithi072.stdout:Package ceph-base-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:13:59.012 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:13:59.013 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:13:59.013 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:13:59.082 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install cephadm 2024-05-21T12:13:59.188 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T12:13:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T12:13:59.190 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T12:13:59.190 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T12:13:59.190 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:13:59.190 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:13:59.368 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-21T12:13:59.402 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 12/12 2024-05-21T12:13:59.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-21T12:13:59.452 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-05-21T12:13:59.477 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): openblas-0.3.21-2.el9.x86_64.rpm 53 kB/s | 35 kB 00:00 2024-05-21T12:13:59.612 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:26 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:13:59.734 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 764 k 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 764 k 2024-05-21T12:13:59.735 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 770 k 2024-05-21T12:13:59.736 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:13:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:19 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:00.097 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:00.098 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 7.4 M 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T12:14:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-05-21T12:14:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:00.101 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 38 M 2024-05-21T12:14:00.101 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 209 M 2024-05-21T12:14:00.101 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:00.127 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.7 MB/s | 4.7 MB 00:01 2024-05-21T12:14:00.336 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-devel-3.9.18-3.el9.x86_64.rpm 276 kB/s | 244 kB 00:00 2024-05-21T12:14:00.422 INFO:teuthology.orchestra.run.smithi072.stdout:cephadm-19.0.0-3665.gb0c764b6.el9.noarch.rpm 1.1 MB/s | 764 kB 00:00 2024-05-21T12:14:00.423 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:00.423 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.1 MB/s | 764 kB 00:00 2024-05-21T12:14:00.423 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:00.428 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:00.429 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:00.435 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:00.435 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:00.508 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:00.836 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.8 MB/s | 5.1 MB 00:01 2024-05-21T12:14:00.986 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:14:01.044 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 273 kB/s | 250 kB 00:00 2024-05-21T12:14:01.082 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:14:01.458 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 203 kB/s | 195 kB 00:00 2024-05-21T12:14:01.460 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:14:01.567 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 749 kB/s | 803 kB 00:01 2024-05-21T12:14:01.706 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/1 2024-05-21T12:14:01.706 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:01.706 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:01.706 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:01.706 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:01.706 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:01.784 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 101 kB/s | 33 kB 00:00 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 12/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 1/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-21T12:14:01.858 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-21T12:14:01.859 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-21T12:14:01.892 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): ceph-mgr-diskprediction-local-19.0.0-36 5.3 MB/s | 7.4 MB 00:01 2024-05-21T12:14:01.917 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 136 kB/s | 18 kB 00:00 2024-05-21T12:14:01.952 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-immutable-object-cache 2024-05-21T12:14:02.003 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.8 MB/s | 16 MB 00:01 2024-05-21T12:14:02.004 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:02.004 INFO:teuthology.orchestra.run.smithi092.stdout:Total 7.4 MB/s | 38 MB 00:05 2024-05-21T12:14:02.159 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 132 kB/s | 35 kB 00:00 2024-05-21T12:14:02.273 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-21T12:14:02.273 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:02.273 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:02.273 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:02.273 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T12:14:02.273 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:02.274 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:02.357 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:02.383 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:02.383 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:02.493 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:29 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:02.617 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:02.618 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:02.618 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2024-05-21T12:14:02.618 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:02.618 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 145 k 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 145 k 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 435 k 2024-05-21T12:14:02.619 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:02.643 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 505 kB/s | 244 kB 00:00 2024-05-21T12:14:02.668 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-rook 2024-05-21T12:14:02.710 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-05-21T12:14:02.811 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:02.811 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:02.852 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 250 kB 00:00 2024-05-21T12:14:02.927 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.7 MB/s | 4.7 MB 00:01 2024-05-21T12:14:03.163 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-immutable-object-cache-19.0.0-3665.gb0c764 266 kB/s | 145 kB 00:00 2024-05-21T12:14:03.164 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:03.164 INFO:teuthology.orchestra.run.smithi072.stdout:Total 265 kB/s | 145 kB 00:00 2024-05-21T12:14:03.164 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:03.173 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:03.173 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:03.204 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:28 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:03.233 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:03.233 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:03.325 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:03.326 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:03.326 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:14:03.326 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:03.326 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:03.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 49 k 2024-05-21T12:14:03.326 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:14:03.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T12:14:03.326 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:03.327 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:03.328 INFO:teuthology.orchestra.run.smithi137.stdout:Install 13 Packages 2024-05-21T12:14:03.328 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:03.328 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 2.2 M 2024-05-21T12:14:03.328 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 26 M 2024-05-21T12:14:03.328 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:03.381 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:03.483 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:14:03.510 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:14:03.510 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:14:03.510 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T12:14:03.510 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:03.642 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-21T12:14:03.684 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-21T12:14:03.787 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-21T12:14:03.827 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-21T12:14:04.006 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-21T12:14:04.035 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-21T12:14:04.084 INFO:teuthology.orchestra.run.smithi137.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 146 kB/s | 19 kB 00:00 2024-05-21T12:14:04.110 INFO:teuthology.orchestra.run.smithi137.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 164 kB/s | 26 kB 00:00 2024-05-21T12:14:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-21T12:14:04.259 INFO:teuthology.orchestra.run.smithi137.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-21T12:14:04.284 INFO:teuthology.orchestra.run.smithi137.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.1 MB/s | 222 kB 00:00 2024-05-21T12:14:04.343 INFO:teuthology.orchestra.run.smithi137.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 921 kB/s | 54 kB 00:00 2024-05-21T12:14:04.368 INFO:teuthology.orchestra.run.smithi137.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 279 kB 00:00 2024-05-21T12:14:04.394 INFO:teuthology.orchestra.run.smithi137.stdout:(7/13): ceph-mgr-rook-19.0.0-3665.gb0c764b6.el9 112 kB/s | 49 kB 00:00 2024-05-21T12:14:04.440 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6. 1/1 2024-05-21T12:14:04.440 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:04.441 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:04.441 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:04.441 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:04.441 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:04.711 INFO:teuthology.orchestra.run.smithi137.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 44 kB/s | 15 kB 00:00 2024-05-21T12:14:04.719 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr 2024-05-21T12:14:04.769 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-05-21T12:14:04.769 INFO:teuthology.orchestra.run.smithi137.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 76 kB/s | 32 kB 00:00 2024-05-21T12:14:04.770 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-21T12:14:04.897 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-21T12:14:04.945 INFO:teuthology.orchestra.run.smithi137.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 338 kB/s | 59 kB 00:00 2024-05-21T12:14:04.970 INFO:teuthology.orchestra.run.smithi137.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 357 kB/s | 206 kB 00:00 2024-05-21T12:14:05.086 INFO:teuthology.orchestra.run.smithi137.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 633 kB/s | 90 kB 00:00 2024-05-21T12:14:05.220 INFO:teuthology.orchestra.run.smithi137.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.0 MB/s | 1.0 MB 00:00 2024-05-21T12:14:05.223 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:05.224 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-05-21T12:14:05.256 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:32 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:05.291 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:05.306 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:05.307 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:05.338 INFO:teuthology.orchestra.run.smithi072.stdout:Package ceph-mgr-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:05.380 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:05.381 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:14:05.381 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:05.391 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:05.392 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:05.448 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-dashboard 2024-05-21T12:14:05.894 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:05.980 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:32 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:06.058 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-21T12:14:06.104 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:06.105 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:06.105 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:14:06.105 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:06.105 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 3.5 M 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 34 k 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 16 k 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T12:14:06.106 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T12:14:06.107 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T12:14:06.107 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:06.107 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:06.107 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:06.107 INFO:teuthology.orchestra.run.smithi072.stdout:Install 12 Packages 2024-05-21T12:14:06.107 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:06.108 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 8.6 M 2024-05-21T12:14:06.108 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 104 M 2024-05-21T12:14:06.108 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:06.113 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-21T12:14:06.138 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-21T12:14:06.169 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-21T12:14:06.206 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-21T12:14:06.247 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-21T12:14:06.340 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-21T12:14:06.450 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-21T12:14:06.498 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-21T12:14:06.777 INFO:teuthology.orchestra.run.smithi072.stdout:(1/12): ceph-prometheus-alerts-19.0.0-3665.gb0c 39 kB/s | 16 kB 00:00 2024-05-21T12:14:06.802 INFO:teuthology.orchestra.run.smithi072.stdout:(2/12): ceph-grafana-dashboards-19.0.0-3665.gb0 76 kB/s | 34 kB 00:00 2024-05-21T12:14:07.188 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-21T12:14:07.212 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-21T12:14:07.218 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 12/12 2024-05-21T12:14:07.254 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-21T12:14:07.288 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-21T12:14:07.336 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-21T12:14:07.366 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 13/13 2024-05-21T12:14:07.394 INFO:teuthology.orchestra.run.smithi072.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 455 kB/s | 269 kB 00:00 2024-05-21T12:14:07.553 INFO:teuthology.orchestra.run.smithi072.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2024-05-21T12:14:07.695 INFO:teuthology.orchestra.run.smithi072.stdout:(5/12): ceph-mgr-dashboard-19.0.0-3665.gb0c764b 2.6 MB/s | 3.5 MB 00:01 2024-05-21T12:14:07.904 INFO:teuthology.orchestra.run.smithi072.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 106 kB/s | 19 kB 00:00 2024-05-21T12:14:07.988 INFO:teuthology.orchestra.run.smithi072.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.2 MB/s | 548 kB 00:00 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 13/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 1/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-21T12:14:08.115 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-21T12:14:08.116 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-21T12:14:08.116 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-21T12:14:08.124 INFO:teuthology.orchestra.run.smithi072.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 9.3 MB/s | 2.0 MB 00:00 2024-05-21T12:14:08.158 INFO:teuthology.orchestra.run.smithi072.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 855 kB/s | 144 kB 00:00 2024-05-21T12:14:08.191 INFO:teuthology.orchestra.run.smithi072.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 458 kB/s | 31 kB 00:00 2024-05-21T12:14:08.226 INFO:teuthology.orchestra.run.smithi072.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.7 MB/s | 188 kB 00:00 2024-05-21T12:14:08.434 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T12:14:08.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T12:14:08.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T12:14:08.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T12:14:08.436 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:08.436 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:08.459 INFO:teuthology.orchestra.run.smithi072.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 811 kB/s | 863 kB 00:01 2024-05-21T12:14:08.464 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:08.464 INFO:teuthology.orchestra.run.smithi072.stdout:Total 3.7 MB/s | 8.6 MB 00:02 2024-05-21T12:14:08.517 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:08.535 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:08.535 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:08.682 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-cephadm 2024-05-21T12:14:08.717 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:08.717 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:09.216 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:34 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:09.321 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:09.337 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:09.338 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:14:09.338 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:09.338 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 143 k 2024-05-21T12:14:09.338 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout:Install 6 Packages 2024-05-21T12:14:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:09.340 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 7.0 M 2024-05-21T12:14:09.340 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 33 M 2024-05-21T12:14:09.340 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:09.494 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-21T12:14:09.539 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-21T12:14:09.586 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-21T12:14:09.617 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-21T12:14:09.733 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-21T12:14:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 12/12 2024-05-21T12:14:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 1/12 2024-05-21T12:14:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-21T12:14:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-21T12:14:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-21T12:14:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-21T12:14:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-21T12:14:09.765 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-21T12:14:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-21T12:14:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-21T12:14:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-21T12:14:09.766 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-21T12:14:09.871 INFO:teuthology.orchestra.run.smithi137.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-05-21T12:14:09.873 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-21T12:14:09.969 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-21T12:14:10.022 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-21T12:14:10.051 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-21T12:14:10.084 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9 10/12 2024-05-21T12:14:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-21T12:14:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:10.169 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T12:14:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T12:14:10.169 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:10.170 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:10.280 INFO:teuthology.orchestra.run.smithi137.stdout:(2/6): ceph-mgr-cephadm-19.0.0-3665.gb0c764b6.e 225 kB/s | 143 kB 00:00 2024-05-21T12:14:10.422 INFO:teuthology.orchestra.run.smithi137.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.7 MB/s | 6.0 MB 00:00 2024-05-21T12:14:10.481 INFO:teuthology.orchestra.run.smithi137.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 288 kB/s | 58 kB 00:00 2024-05-21T12:14:10.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-rook 2024-05-21T12:14:10.589 INFO:teuthology.orchestra.run.smithi137.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 764 kB/s | 548 kB 00:00 2024-05-21T12:14:10.606 INFO:teuthology.orchestra.run.smithi137.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 278 kB/s | 51 kB 00:00 2024-05-21T12:14:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:10.608 INFO:teuthology.orchestra.run.smithi137.stdout:Total 5.5 MB/s | 7.0 MB 00:01 2024-05-21T12:14:10.680 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:10.691 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:10.691 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:10.787 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:10.788 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:11.087 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:25 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:11.096 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:11.135 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el 11/12 2024-05-21T12:14:11.179 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 12/12 2024-05-21T12:14:11.210 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:14:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:11.211 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 49 k 2024-05-21T12:14:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T12:14:11.212 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:11.213 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:11.213 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:11.213 INFO:teuthology.orchestra.run.smithi092.stdout:Install 13 Packages 2024-05-21T12:14:11.213 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:11.213 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.2 M 2024-05-21T12:14:11.214 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 26 M 2024-05-21T12:14:11.214 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:11.253 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-21T12:14:11.309 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-21T12:14:11.784 INFO:teuthology.orchestra.run.smithi092.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 143 kB/s | 19 kB 00:00 2024-05-21T12:14:11.809 INFO:teuthology.orchestra.run.smithi092.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 161 kB/s | 26 kB 00:00 2024-05-21T12:14:11.826 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-21T12:14:11.902 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-21T12:14:11.934 INFO:teuthology.orchestra.run.smithi092.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-21T12:14:11.959 INFO:teuthology.orchestra.run.smithi092.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-05-21T12:14:11.974 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-05-21T12:14:11.984 INFO:teuthology.orchestra.run.smithi092.stdout:(5/13): ceph-mgr-rook-19.0.0-3665.gb0c764b6.el9 147 kB/s | 49 kB 00:00 2024-05-21T12:14:12.003 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 6/6 2024-05-21T12:14:12.010 INFO:teuthology.orchestra.run.smithi092.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-05-21T12:14:12.035 INFO:teuthology.orchestra.run.smithi092.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-05-21T12:14:12.069 INFO:teuthology.orchestra.run.smithi092.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 383 kB/s | 32 kB 00:00 2024-05-21T12:14:12.094 INFO:teuthology.orchestra.run.smithi092.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 178 kB/s | 15 kB 00:00 2024-05-21T12:14:12.152 INFO:teuthology.orchestra.run.smithi092.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-21T12:14:12.178 INFO:teuthology.orchestra.run.smithi092.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 1.4 MB/s | 206 kB 00:00 2024-05-21T12:14:12.211 INFO:teuthology.orchestra.run.smithi092.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-05-21T12:14:12.311 INFO:teuthology.orchestra.run.smithi092.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.2 MB/s | 1.0 MB 00:00 2024-05-21T12:14:12.312 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:12.312 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-05-21T12:14:12.376 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:12.391 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:12.391 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:12.475 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:12.475 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:12.819 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 6/6 2024-05-21T12:14:12.819 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/6 2024-05-21T12:14:12.819 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-21T12:14:12.820 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-05-21T12:14:12.820 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-21T12:14:12.820 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-21T12:14:12.972 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T12:14:13.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T12:14:13.079 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:13.079 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-21T12:14:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-21T12:14:13.245 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 12/12 2024-05-21T12:14:13.245 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el 1/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noa 2/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9 3/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-21T12:14:13.246 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-21T12:14:13.251 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-21T12:14:13.289 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-21T12:14:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-21T12:14:13.359 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-fuse 2024-05-21T12:14:13.433 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T12:14:13.538 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T12:14:13.539 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T12:14:13.539 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T12:14:13.539 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T12:14:13.539 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T12:14:13.539 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:13.539 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:13.552 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-21T12:14:13.607 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-21T12:14:13.868 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-21T12:14:13.894 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:38 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:14.016 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:14.017 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:14:14.017 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:14.018 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 890 k 2024-05-21T12:14:14.018 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:14:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T12:14:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:14.018 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:14.019 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:14.019 INFO:teuthology.orchestra.run.smithi137.stdout:Install 2 Packages 2024-05-21T12:14:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:14.019 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 970 k 2024-05-21T12:14:14.019 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 2.8 M 2024-05-21T12:14:14.019 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:14.336 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-21T12:14:14.374 INFO:teuthology.orchestra.run.smithi137.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 397 kB/s | 80 kB 00:00 2024-05-21T12:14:14.387 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-21T12:14:14.395 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:41 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-21T12:14:14.450 INFO:teuthology.orchestra.run.smithi137.stdout:(2/2): ceph-fuse-19.0.0-3665.gb0c764b6.el9.x86_ 3.1 MB/s | 890 kB 00:00 2024-05-21T12:14:14.450 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:14.451 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.2 MB/s | 970 kB 00:00 2024-05-21T12:14:14.463 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:14.469 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-21T12:14:14.485 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:14.485 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:14.499 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 13/13 2024-05-21T12:14:14.533 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:14.534 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-21T12:14:14.534 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:14:14.534 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-21T12:14:14.534 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:14.535 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 7.4 M 2024-05-21T12:14:14.535 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:14:14.535 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T12:14:14.535 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T12:14:14.535 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T12:14:14.535 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T12:14:14.536 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T12:14:14.536 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T12:14:14.536 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T12:14:14.536 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T12:14:14.536 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T12:14:14.536 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T12:14:14.537 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T12:14:14.537 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:14.537 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:14.537 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-21T12:14:14.537 INFO:teuthology.orchestra.run.smithi072.stdout:Install 12 Packages 2024-05-21T12:14:14.537 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:14.538 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 38 M 2024-05-21T12:14:14.538 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 209 M 2024-05-21T12:14:14.539 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:14.562 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:14.751 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:14.884 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-21T12:14:14.948 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/2 2024-05-21T12:14:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 13/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 1/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-21T12:14:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-21T12:14:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-21T12:14:15.339 INFO:teuthology.orchestra.run.smithi072.stdout:(1/12): ceph-mgr-diskprediction-local-19.0.0-36 11 MB/s | 7.4 MB 00:00 2024-05-21T12:14:15.514 INFO:teuthology.orchestra.run.smithi072.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 187 kB/s | 33 kB 00:00 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T12:14:15.524 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T12:14:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T12:14:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T12:14:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T12:14:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T12:14:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T12:14:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T12:14:15.525 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:15.525 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/2 2024-05-21T12:14:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/2 2024-05-21T12:14:15.690 INFO:teuthology.orchestra.run.smithi072.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 194 kB/s | 195 kB 00:01 2024-05-21T12:14:15.782 INFO:teuthology.orchestra.run.smithi072.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 199 kB/s | 18 kB 00:00 2024-05-21T12:14:15.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-cephadm 2024-05-21T12:14:15.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-21T12:14:15.867 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:15.867 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:15.867 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-21T12:14:15.867 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:15.868 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:15.891 INFO:teuthology.orchestra.run.smithi072.stdout:(5/12): libgfortran-11.4.1-3.el9.x86_64.rpm 665 kB/s | 803 kB 00:01 2024-05-21T12:14:15.908 INFO:teuthology.orchestra.run.smithi072.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 281 kB/s | 35 kB 00:00 2024-05-21T12:14:16.125 INFO:teuthology.orchestra.run.smithi072.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-05-21T12:14:16.140 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-volume 2024-05-21T12:14:16.201 INFO:teuthology.orchestra.run.smithi072.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.4 MB/s | 3.0 MB 00:00 2024-05-21T12:14:16.310 INFO:teuthology.orchestra.run.smithi072.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 250 kB 00:00 2024-05-21T12:14:16.338 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:30 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:16.460 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:16.460 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 143 k 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T12:14:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:16.462 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:16.462 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:16.462 INFO:teuthology.orchestra.run.smithi092.stdout:Install 6 Packages 2024-05-21T12:14:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:16.462 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 7.0 M 2024-05-21T12:14:16.462 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 33 M 2024-05-21T12:14:16.463 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:16.679 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:41 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:16.807 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:16.807 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:16.807 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-21T12:14:16.807 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:16.807 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 273 k 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout:Install 2 Packages 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 350 k 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 1.5 M 2024-05-21T12:14:16.808 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:17.254 INFO:teuthology.orchestra.run.smithi137.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 310 kB/s | 77 kB 00:00 2024-05-21T12:14:17.278 INFO:teuthology.orchestra.run.smithi092.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 875 kB/s | 248 kB 00:00 2024-05-21T12:14:17.441 INFO:teuthology.orchestra.run.smithi092.stdout:(2/6): ceph-mgr-cephadm-19.0.0-3665.gb0c764b6.e 321 kB/s | 143 kB 00:00 2024-05-21T12:14:17.460 INFO:teuthology.orchestra.run.smithi072.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 3.1 MB/s | 4.7 MB 00:01 2024-05-21T12:14:17.487 INFO:teuthology.orchestra.run.smithi137.stdout:(2/2): ceph-volume-19.0.0-3665.gb0c764b6.el9.no 566 kB/s | 273 kB 00:00 2024-05-21T12:14:17.488 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:17.488 INFO:teuthology.orchestra.run.smithi137.stdout:Total 513 kB/s | 350 kB 00:00 2024-05-21T12:14:17.501 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:17.510 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:17.510 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:17.541 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:17.542 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:17.596 INFO:teuthology.orchestra.run.smithi092.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 373 kB/s | 58 kB 00:00 2024-05-21T12:14:17.679 INFO:teuthology.orchestra.run.smithi092.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 612 kB/s | 51 kB 00:00 2024-05-21T12:14:17.685 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:17.746 INFO:teuthology.orchestra.run.smithi092.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-05-21T12:14:17.844 INFO:teuthology.orchestra.run.smithi072.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.0 MB/s | 5.1 MB 00:01 2024-05-21T12:14:17.881 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-21T12:14:17.907 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2/2 2024-05-21T12:14:17.923 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2/2 2024-05-21T12:14:17.923 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:14:17.923 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T12:14:17.923 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:18.430 INFO:teuthology.orchestra.run.smithi092.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.2 MB/s | 6.0 MB 00:01 2024-05-21T12:14:18.431 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:18.431 INFO:teuthology.orchestra.run.smithi092.stdout:Total 3.5 MB/s | 7.0 MB 00:01 2024-05-21T12:14:18.503 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:18.514 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:18.514 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:18.612 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:18.613 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:18.693 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 1/2 2024-05-21T12:14:18.918 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-21T12:14:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:19.001 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:19.001 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:19.001 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T12:14:19.001 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:19.001 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:19.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-21T12:14:19.149 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-21T12:14:19.288 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librados-devel 2024-05-21T12:14:19.361 INFO:teuthology.orchestra.run.smithi072.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:03 2024-05-21T12:14:19.362 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:19.362 INFO:teuthology.orchestra.run.smithi072.stdout:Total 7.9 MB/s | 38 MB 00:04 2024-05-21T12:14:19.675 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-21T12:14:19.718 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:19.746 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:19.746 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:19.760 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-21T12:14:19.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-05-21T12:14:19.832 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:44 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:19.852 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 6/6 2024-05-21T12:14:19.963 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:19.964 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-05-21T12:14:19.964 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:19.964 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 127 k 2024-05-21T12:14:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:19.964 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:19.964 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:19.965 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-05-21T12:14:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:19.965 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 127 k 2024-05-21T12:14:19.965 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 456 k 2024-05-21T12:14:19.965 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:20.187 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:20.187 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:20.478 INFO:teuthology.orchestra.run.smithi137.stdout:librados-devel-19.0.0-3665.gb0c764b6.el9.x86_64 247 kB/s | 127 kB 00:00 2024-05-21T12:14:20.479 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:20.479 INFO:teuthology.orchestra.run.smithi137.stdout:Total 246 kB/s | 127 kB 00:00 2024-05-21T12:14:20.479 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:20.484 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:20.485 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:20.548 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:20.549 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:20.549 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 6/6 2024-05-21T12:14:20.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/6 2024-05-21T12:14:20.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-21T12:14:20.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-05-21T12:14:20.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-21T12:14:20.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-21T12:14:20.702 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:20.809 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T12:14:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T12:14:20.825 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:20.825 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:20.941 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:20.942 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 920 kB/s | 16 MB 00:18 2024-05-21T12:14:20.944 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:20.944 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.8 MB/s | 38 MB 00:20 2024-05-21T12:14:21.010 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-21T12:14:21.053 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-21T12:14:21.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-fuse 2024-05-21T12:14:21.153 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-21T12:14:21.197 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-21T12:14:21.209 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:21.311 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:21.340 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:21.340 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:21.375 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-21T12:14:21.405 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-21T12:14:21.445 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:21.445 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:21.445 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:21.445 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:21.445 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:21.445 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:21.501 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-21T12:14:21.614 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:35 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:21.679 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install libcephfs2 2024-05-21T12:14:21.736 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 890 k 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:21.737 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-05-21T12:14:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:21.738 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 970 k 2024-05-21T12:14:21.738 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 2.8 M 2024-05-21T12:14:21.738 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:22.156 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 324 kB/s | 80 kB 00:00 2024-05-21T12:14:22.205 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-21T12:14:22.221 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:47 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:22.303 INFO:teuthology.orchestra.run.smithi137.stdout:Package libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:22.335 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-21T12:14:22.346 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:22.347 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:14:22.348 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:22.439 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install libcephfs-devel 2024-05-21T12:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-21T12:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-21T12:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-21T12:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-21T12:14:22.994 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:47 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:23.006 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-fuse-19.0.0-3665.gb0c764b6.el9.x86_ 810 kB/s | 890 kB 00:01 2024-05-21T12:14:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:Total 764 kB/s | 970 kB 00:01 2024-05-21T12:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-21T12:14:23.020 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-21T12:14:23.044 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:23.044 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:23.117 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:23.117 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-05-21T12:14:23.117 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:23.117 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 32 k 2024-05-21T12:14:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:23.117 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:23.118 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:23.118 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-05-21T12:14:23.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:23.118 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 32 k 2024-05-21T12:14:23.118 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 142 k 2024-05-21T12:14:23.118 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-21T12:14:23.124 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:23.124 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:23.440 INFO:teuthology.orchestra.run.smithi137.stdout:libcephfs-devel-19.0.0-3665.gb0c764b6.el9.x86_6 98 kB/s | 32 kB 00:00 2024-05-21T12:14:23.440 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-21T12:14:23.441 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:23.441 INFO:teuthology.orchestra.run.smithi137.stdout:Total 98 kB/s | 32 kB 00:00 2024-05-21T12:14:23.441 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:23.444 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:23.445 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:23.457 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:23.458 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:23.505 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/2 2024-05-21T12:14:23.540 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:23.569 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-21T12:14:23.646 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-21T12:14:23.934 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-21T12:14:24.044 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:24.185 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/2 2024-05-21T12:14:24.185 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/2 2024-05-21T12:14:24.272 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:24.273 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:24.273 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:24.273 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:24.273 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:24.273 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:24.447 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librados2 2024-05-21T12:14:24.456 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-21T12:14:24.456 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:24.456 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:24.456 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-21T12:14:24.456 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:24.456 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:24.630 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-21T12:14:24.656 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 12/12 2024-05-21T12:14:24.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-volume 2024-05-21T12:14:24.978 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:49 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:25.058 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:25.099 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:25.100 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:14:25.101 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:25.173 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librbd1 2024-05-21T12:14:25.258 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-21T12:14:25.284 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:39 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:25.410 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:25.410 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 273 k 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-05-21T12:14:25.411 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:25.412 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 350 k 2024-05-21T12:14:25.412 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 1.5 M 2024-05-21T12:14:25.412 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:25.705 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:50 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:25.786 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:25.827 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:25.828 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:14:25.828 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:25.839 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 305 kB/s | 77 kB 00:00 2024-05-21T12:14:25.904 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rados 2024-05-21T12:14:25.994 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-volume-19.0.0-3665.gb0c764b6.el9.no 664 kB/s | 273 kB 00:00 2024-05-21T12:14:25.995 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:25.995 INFO:teuthology.orchestra.run.smithi092.stdout:Total 600 kB/s | 350 kB 00:00 2024-05-21T12:14:26.007 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:26.016 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:26.016 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:26.051 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:26.051 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:26.199 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:26.356 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-21T12:14:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 12/12 2024-05-21T12:14:26.391 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-21T12:14:26.430 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2/2 2024-05-21T12:14:26.431 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:51 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2/2 2024-05-21T12:14:26.445 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:14:26.445 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T12:14:26.445 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:26.512 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:26.554 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:26.555 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:14:26.555 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:26.629 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rgw 2024-05-21T12:14:27.097 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 1/2 2024-05-21T12:14:27.161 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:52 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:27.241 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:27.283 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:27.284 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:14:27.284 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:27.359 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-cephfs 2024-05-21T12:14:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-21T12:14:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:27.408 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T12:14:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:27.408 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:27.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados-devel 2024-05-21T12:14:27.895 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:52 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:27.974 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:28.016 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:28.017 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:14:28.017 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:28.088 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rbd 2024-05-21T12:14:28.245 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:42 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:28.371 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 127 k 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-21T12:14:28.372 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:28.373 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 127 k 2024-05-21T12:14:28.373 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 456 k 2024-05-21T12:14:28.373 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:28.613 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:53 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:28.683 INFO:teuthology.orchestra.run.smithi092.stdout:librados-devel-19.0.0-3665.gb0c764b6.el9.x86_64 408 kB/s | 127 kB 00:00 2024-05-21T12:14:28.684 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:28.684 INFO:teuthology.orchestra.run.smithi092.stdout:Total 406 kB/s | 127 kB 00:00 2024-05-21T12:14:28.684 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:28.690 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:28.690 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:28.693 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:28.735 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:28.736 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-21T12:14:28.736 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:28.753 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:28.753 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:28.782 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 12/12 2024-05-21T12:14:28.782 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 1/12 2024-05-21T12:14:28.782 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-21T12:14:28.782 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-21T12:14:28.782 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-21T12:14:28.783 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-21T12:14:28.783 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-21T12:14:28.783 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-21T12:14:28.783 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-21T12:14:28.783 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-21T12:14:28.783 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-21T12:14:28.783 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-21T12:14:28.810 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-fuse 2024-05-21T12:14:28.887 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 12/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c76 1/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-21T12:14:28.888 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-21T12:14:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T12:14:29.172 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T12:14:29.173 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T12:14:29.173 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T12:14:29.173 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T12:14:29.173 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T12:14:29.173 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:29.173 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:29.235 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-21T12:14:29.235 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:29.235 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:14:29.235 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:29.235 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T12:14:29.235 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T12:14:29.235 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:29.236 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:29.340 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:54 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:29.464 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:29.464 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 85 k 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 85 k 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 231 k 2024-05-21T12:14:29.465 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:29.471 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:29.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-rook 2024-05-21T12:14:29.660 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-rook 2024-05-21T12:14:29.721 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:29.721 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:29.721 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:29.721 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:29.721 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:29.721 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:29.818 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-fuse-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 242 kB/s | 85 kB 00:00 2024-05-21T12:14:29.818 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:29.818 INFO:teuthology.orchestra.run.smithi137.stdout:Total 241 kB/s | 85 kB 00:00 2024-05-21T12:14:29.818 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:29.825 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:29.826 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:29.890 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:29.890 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:29.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs2 2024-05-21T12:14:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:30.132 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:30.198 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:57 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:30.203 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:50 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:30.320 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 49 k 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T12:14:30.321 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:30.322 INFO:teuthology.orchestra.run.smithi072.stdout:Install 13 Packages 2024-05-21T12:14:30.323 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:30.323 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 2.2 M 2024-05-21T12:14:30.323 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 26 M 2024-05-21T12:14:30.323 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:30.331 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:30.332 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:30.332 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:14:30.332 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:30.332 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:30.332 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 49 k 2024-05-21T12:14:30.332 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T12:14:30.333 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:30.334 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:30.334 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:30.334 INFO:teuthology.orchestra.run.smithi045.stdout:Install 13 Packages 2024-05-21T12:14:30.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:30.334 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 2.2 M 2024-05-21T12:14:30.335 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 26 M 2024-05-21T12:14:30.335 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:30.502 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:44 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:30.583 INFO:teuthology.orchestra.run.smithi092.stdout:Package libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:30.626 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:30.627 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:14:30.628 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:30.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs-devel 2024-05-21T12:14:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:30.764 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:30.764 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:30.836 INFO:teuthology.orchestra.run.smithi045.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 138 kB/s | 19 kB 00:00 2024-05-21T12:14:30.861 INFO:teuthology.orchestra.run.smithi045.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 156 kB/s | 26 kB 00:00 2024-05-21T12:14:30.862 INFO:teuthology.orchestra.run.smithi072.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 241 kB/s | 26 kB 00:00 2024-05-21T12:14:30.887 INFO:teuthology.orchestra.run.smithi072.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 145 kB/s | 19 kB 00:00 2024-05-21T12:14:30.954 INFO:teuthology.orchestra.run.smithi072.stdout:(3/13): ceph-mgr-rook-19.0.0-3665.gb0c764b6.el9 247 kB/s | 49 kB 00:00 2024-05-21T12:14:30.986 INFO:teuthology.orchestra.run.smithi045.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-21T12:14:31.011 INFO:teuthology.orchestra.run.smithi045.stdout:(4/13): ceph-mgr-rook-19.0.0-3665.gb0c764b6.el9 157 kB/s | 49 kB 00:00 2024-05-21T12:14:31.013 INFO:teuthology.orchestra.run.smithi072.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-05-21T12:14:31.018 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-mirror 2024-05-21T12:14:31.039 INFO:teuthology.orchestra.run.smithi072.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-21T12:14:31.064 INFO:teuthology.orchestra.run.smithi072.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-05-21T12:14:31.087 INFO:teuthology.orchestra.run.smithi045.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-05-21T12:14:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:(6/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 783 kB/s | 222 kB 00:00 2024-05-21T12:14:31.162 INFO:teuthology.orchestra.run.smithi045.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 355 kB/s | 54 kB 00:00 2024-05-21T12:14:31.179 INFO:teuthology.orchestra.run.smithi045.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 348 kB/s | 32 kB 00:00 2024-05-21T12:14:31.196 INFO:teuthology.orchestra.run.smithi045.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 198 kB/s | 15 kB 00:00 2024-05-21T12:14:31.198 INFO:teuthology.orchestra.run.smithi072.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-05-21T12:14:31.224 INFO:teuthology.orchestra.run.smithi072.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 174 kB/s | 32 kB 00:00 2024-05-21T12:14:31.249 INFO:teuthology.orchestra.run.smithi072.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 93 kB/s | 15 kB 00:00 2024-05-21T12:14:31.271 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:45 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:31.280 INFO:teuthology.orchestra.run.smithi045.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 711 kB/s | 59 kB 00:00 2024-05-21T12:14:31.301 INFO:teuthology.orchestra.run.smithi072.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 2.0 MB/s | 206 kB 00:00 2024-05-21T12:14:31.347 INFO:teuthology.orchestra.run.smithi045.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-05-21T12:14:31.372 INFO:teuthology.orchestra.run.smithi045.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 982 kB/s | 206 kB 00:00 2024-05-21T12:14:31.392 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:31.393 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-05-21T12:14:31.393 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:31.393 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 32 k 2024-05-21T12:14:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:31.393 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:31.393 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:31.394 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-21T12:14:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:31.394 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 32 k 2024-05-21T12:14:31.394 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 142 k 2024-05-21T12:14:31.394 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:31.410 INFO:teuthology.orchestra.run.smithi072.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 5.5 MB/s | 1.0 MB 00:00 2024-05-21T12:14:31.435 INFO:teuthology.orchestra.run.smithi072.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 317 kB/s | 59 kB 00:00 2024-05-21T12:14:31.461 INFO:teuthology.orchestra.run.smithi072.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 564 kB/s | 90 kB 00:00 2024-05-21T12:14:31.464 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:31.464 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-05-21T12:14:31.529 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:31.544 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:31.545 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:31.551 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:56 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:31.631 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:31.631 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:31.648 INFO:teuthology.orchestra.run.smithi045.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.2 MB/s | 1.0 MB 00:00 2024-05-21T12:14:31.648 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:31.648 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-05-21T12:14:31.674 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:31.675 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:31.675 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-05-21T12:14:31.675 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:31.675 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:31.675 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.1 M 2024-05-21T12:14:31.675 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:31.675 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:31.675 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:31.676 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-05-21T12:14:31.676 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:31.676 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 3.1 M 2024-05-21T12:14:31.676 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 13 M 2024-05-21T12:14:31.676 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:31.712 INFO:teuthology.orchestra.run.smithi092.stdout:libcephfs-devel-19.0.0-3665.gb0c764b6.el9.x86_6 100 kB/s | 32 kB 00:00 2024-05-21T12:14:31.712 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:31.713 INFO:teuthology.orchestra.run.smithi092.stdout:Total 99 kB/s | 32 kB 00:00 2024-05-21T12:14:31.713 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:31.716 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:31.716 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:31.719 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:31.733 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:31.733 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:31.736 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:31.736 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:31.816 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:31.831 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:31.934 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:32.131 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:32.295 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:32.300 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-21T12:14:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:14:32.356 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-21T12:14:32.412 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-21T12:14:32.450 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-21T12:14:32.493 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-21T12:14:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:32.502 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:32.502 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-21T12:14:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-21T12:14:32.587 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-21T12:14:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-21T12:14:32.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados2 2024-05-21T12:14:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-21T12:14:32.698 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-21T12:14:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-21T12:14:32.726 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-mirror-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 2.9 MB/s | 3.1 MB 00:01 2024-05-21T12:14:32.726 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:32.726 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.9 MB/s | 3.1 MB 00:01 2024-05-21T12:14:32.726 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:32.736 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:32.736 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:32.755 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-21T12:14:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-21T12:14:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:32.818 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-21T12:14:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-21T12:14:33.203 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:47 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:33.250 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:33.283 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:33.317 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:33.324 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:33.325 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:14:33.326 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:33.342 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:33.343 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:14:33.343 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T12:14:33.343 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T12:14:33.343 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T12:14:33.343 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:33.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librbd1 2024-05-21T12:14:33.495 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-21T12:14:33.549 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-21T12:14:33.584 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-21T12:14:33.641 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-21T12:14:33.673 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 13/13 2024-05-21T12:14:33.762 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-21T12:14:33.816 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-21T12:14:33.850 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-21T12:14:33.899 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-21T12:14:33.921 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:47 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:33.928 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 13/13 2024-05-21T12:14:34.000 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:34.041 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:34.042 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:14:34.043 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:34.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rados 2024-05-21T12:14:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:34.363 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:34.363 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:34.604 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-nbd 2024-05-21T12:14:34.633 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:48 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 13/13 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 1/13 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-21T12:14:34.648 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-21T12:14:34.649 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-21T12:14:34.649 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-21T12:14:34.649 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-21T12:14:34.649 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-21T12:14:34.712 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:34.719 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 13/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 1/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-21T12:14:34.720 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-21T12:14:34.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-21T12:14:34.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-21T12:14:34.721 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-21T12:14:34.754 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:34.755 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:14:34.755 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:34.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rgw 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T12:14:34.975 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:34.976 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:35.038 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-21T12:14:35.038 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:35.038 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:35.038 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:35.039 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T12:14:35.039 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T12:14:35.039 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T12:14:35.039 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T12:14:35.039 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T12:14:35.039 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T12:14:35.039 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T12:14:35.039 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T12:14:35.040 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T12:14:35.040 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T12:14:35.040 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T12:14:35.040 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T12:14:35.040 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:35.040 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:35.152 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:00 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:35.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-cephadm 2024-05-21T12:14:35.279 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 171 k 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-05-21T12:14:35.280 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:35.281 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 171 k 2024-05-21T12:14:35.281 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 494 k 2024-05-21T12:14:35.281 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:35.357 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-cephadm 2024-05-21T12:14:35.360 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:49 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:35.440 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:35.482 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:35.483 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:14:35.483 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:35.507 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-nbd-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 756 kB/s | 171 kB 00:00 2024-05-21T12:14:35.508 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:35.508 INFO:teuthology.orchestra.run.smithi137.stdout:Total 751 kB/s | 171 kB 00:00 2024-05-21T12:14:35.508 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:35.516 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:35.516 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:35.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-cephfs 2024-05-21T12:14:35.579 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:35.580 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:35.728 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:35.790 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:55 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:35.838 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:35.891 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:02 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:35.918 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 143 k 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T12:14:35.919 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T12:14:35.920 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T12:14:35.920 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:35.920 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:35.920 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:35.920 INFO:teuthology.orchestra.run.smithi045.stdout:Install 6 Packages 2024-05-21T12:14:35.920 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:35.921 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 7.0 M 2024-05-21T12:14:35.921 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 33 M 2024-05-21T12:14:35.921 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:36.011 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 143 k 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T12:14:36.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T12:14:36.013 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T12:14:36.013 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:36.013 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:36.013 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:36.013 INFO:teuthology.orchestra.run.smithi072.stdout:Install 6 Packages 2024-05-21T12:14:36.013 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:36.013 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 7.0 M 2024-05-21T12:14:36.013 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 33 M 2024-05-21T12:14:36.014 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:36.080 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:50 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:36.159 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:36.200 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:36.201 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:14:36.201 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:36.233 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:36.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rbd 2024-05-21T12:14:36.497 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:36.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:36.497 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-21T12:14:36.497 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:36.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:36.497 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:36.521 INFO:teuthology.orchestra.run.smithi072.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 731 kB/s | 248 kB 00:00 2024-05-21T12:14:36.534 INFO:teuthology.orchestra.run.smithi045.stdout:(1/6): ceph-mgr-cephadm-19.0.0-3665.gb0c764b6.e 621 kB/s | 143 kB 00:00 2024-05-21T12:14:36.651 INFO:teuthology.orchestra.run.smithi045.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 715 kB/s | 248 kB 00:00 2024-05-21T12:14:36.672 INFO:teuthology.orchestra.run.smithi072.stdout:(2/6): ceph-mgr-cephadm-19.0.0-3665.gb0c764b6.e 292 kB/s | 143 kB 00:00 2024-05-21T12:14:36.734 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install nvme-cli 2024-05-21T12:14:36.804 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:50 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:36.883 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:36.901 INFO:teuthology.orchestra.run.smithi045.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 232 kB/s | 58 kB 00:00 2024-05-21T12:14:36.925 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:36.926 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-05-21T12:14:36.927 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:36.985 INFO:teuthology.orchestra.run.smithi045.stdout:(4/6): python3-babel-2.9.1-2.el9.noarch.rpm 8.8 MB/s | 6.0 MB 00:00 2024-05-21T12:14:37.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-fuse 2024-05-21T12:14:37.010 INFO:teuthology.orchestra.run.smithi045.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-05-21T12:14:37.035 INFO:teuthology.orchestra.run.smithi045.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 382 kB/s | 51 kB 00:00 2024-05-21T12:14:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:37.038 INFO:teuthology.orchestra.run.smithi045.stdout:Total 6.2 MB/s | 7.0 MB 00:01 2024-05-21T12:14:37.039 INFO:teuthology.orchestra.run.smithi072.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 157 kB/s | 58 kB 00:00 2024-05-21T12:14:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:37.124 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:37.124 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:37.156 INFO:teuthology.orchestra.run.smithi072.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 864 kB/s | 548 kB 00:00 2024-05-21T12:14:37.227 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:37.228 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:37.233 INFO:teuthology.orchestra.run.smithi072.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 263 kB/s | 51 kB 00:00 2024-05-21T12:14:37.273 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:02 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:14:37.357 INFO:teuthology.orchestra.run.smithi137.stdout:Package nvme-cli-2.6-5.el9.x86_64 is already installed. 2024-05-21T12:14:37.401 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-21T12:14:37.402 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:37.402 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-05-21T12:14:37.402 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:37.402 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-05-21T12:14:37.402 INFO:teuthology.orchestra.run.smithi137.stdout: libnvme x86_64 1.9-1.el9 baseos 104 k 2024-05-21T12:14:37.402 INFO:teuthology.orchestra.run.smithi137.stdout: nvme-cli x86_64 2.9.1-2.el9 baseos 906 k 2024-05-21T12:14:37.402 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:37.402 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-21T12:14:37.403 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-21T12:14:37.403 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 2 Packages 2024-05-21T12:14:37.403 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:37.403 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 1.0 M 2024-05-21T12:14:37.403 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-21T12:14:37.542 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:14:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:51 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:37.674 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 85 k 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:37.675 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 85 k 2024-05-21T12:14:37.676 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 231 k 2024-05-21T12:14:37.676 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:37.699 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-21T12:14:37.755 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-21T12:14:37.756 INFO:teuthology.orchestra.run.smithi137.stdout:(1/2): libnvme-1.9-1.el9.x86_64.rpm 431 kB/s | 104 kB 00:00 2024-05-21T12:14:37.880 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-fuse-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 417 kB/s | 85 kB 00:00 2024-05-21T12:14:37.881 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:37.881 INFO:teuthology.orchestra.run.smithi092.stdout:Total 415 kB/s | 85 kB 00:00 2024-05-21T12:14:37.881 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:37.888 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:37.888 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:37.895 INFO:teuthology.orchestra.run.smithi137.stdout:(2/2): nvme-cli-2.9.1-2.el9.x86_64.rpm 2.3 MB/s | 906 kB 00:00 2024-05-21T12:14:37.895 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:37.895 INFO:teuthology.orchestra.run.smithi137.stdout:Total 2.0 MB/s | 1.0 MB 00:00 2024-05-21T12:14:37.923 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-21T12:14:37.935 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:14:37.936 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-21T12:14:37.951 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:37.952 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:38.042 INFO:teuthology.orchestra.run.smithi072.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-05-21T12:14:38.043 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:38.043 INFO:teuthology.orchestra.run.smithi072.stdout:Total 3.4 MB/s | 7.0 MB 00:02 2024-05-21T12:14:38.044 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:14:38.045 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-21T12:14:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:38.116 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:38.128 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:38.128 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:38.211 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:38.224 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:38.224 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:38.289 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-21T12:14:38.309 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-21T12:14:38.393 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-21T12:14:38.474 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-05-21T12:14:38.500 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 6/6 2024-05-21T12:14:38.526 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:38.570 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libnvme-1.9-1.el9.x86_64 1/4 2024-05-21T12:14:38.580 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:38.659 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : nvme-cli-2.9.1-2.el9.x86_64 2/4 2024-05-21T12:14:38.700 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-21T12:14:38.754 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-21T12:14:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:38.828 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:38.828 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:38.828 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:38.941 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: nvme-cli-2.9.1-2.el9.x86_64 2/4 2024-05-21T12:14:38.977 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : nvme-cli-2.6-5.el9.x86_64 3/4 2024-05-21T12:14:39.072 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libnvme-1.6-1.el9.x86_64 4/4 2024-05-21T12:14:39.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-mirror 2024-05-21T12:14:39.279 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 6/6 2024-05-21T12:14:39.279 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/6 2024-05-21T12:14:39.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-21T12:14:39.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-05-21T12:14:39.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-21T12:14:39.280 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-21T12:14:39.284 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-21T12:14:39.377 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-21T12:14:39.448 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-05-21T12:14:39.496 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 6/6 2024-05-21T12:14:39.548 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-21T12:14:39.548 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:39.549 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:39.623 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:53 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:39.748 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.1 M 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-21T12:14:39.749 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:39.750 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 3.1 M 2024-05-21T12:14:39.750 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 13 M 2024-05-21T12:14:39.750 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:39.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-fuse 2024-05-21T12:14:39.863 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libnvme-1.6-1.el9.x86_64 4/4 2024-05-21T12:14:39.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libnvme-1.9-1.el9.x86_64 1/4 2024-05-21T12:14:39.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libnvme-1.6-1.el9.x86_64 2/4 2024-05-21T12:14:39.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : nvme-cli-2.9.1-2.el9.x86_64 3/4 2024-05-21T12:14:40.114 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-mirror-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 8.5 MB/s | 3.1 MB 00:00 2024-05-21T12:14:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:Total 8.5 MB/s | 3.1 MB 00:00 2024-05-21T12:14:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:40.124 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:40.124 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:40.156 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : nvme-cli-2.6-5.el9.x86_64 4/4 2024-05-21T12:14:40.156 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:40.156 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-05-21T12:14:40.157 INFO:teuthology.orchestra.run.smithi137.stdout: libnvme-1.9-1.el9.x86_64 nvme-cli-2.9.1-2.el9.x86_64 2024-05-21T12:14:40.157 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:14:40.157 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-21T12:14:40.203 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:40.204 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:40.354 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:00 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:40.481 DEBUG:teuthology.parallel:result is None 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 890 k 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:40.482 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:40.483 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:40.483 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2024-05-21T12:14:40.483 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:40.483 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 970 k 2024-05-21T12:14:40.483 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.8 M 2024-05-21T12:14:40.483 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:40.582 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 6/6 2024-05-21T12:14:40.582 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 1/6 2024-05-21T12:14:40.582 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-21T12:14:40.582 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-05-21T12:14:40.582 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-21T12:14:40.582 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-21T12:14:40.629 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:40.715 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:14:40.715 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T12:14:40.716 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T12:14:40.716 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T12:14:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:40.830 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): ceph-fuse-19.0.0-3665.gb0c764b6.el9.x86_ 2.9 MB/s | 890 kB 00:00 2024-05-21T12:14:40.835 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-21T12:14:40.835 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:40.835 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:40.835 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:40.836 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T12:14:40.836 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T12:14:40.836 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T12:14:40.836 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T12:14:40.836 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T12:14:40.836 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:40.836 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:41.039 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 158 kB/s | 80 kB 00:00 2024-05-21T12:14:41.039 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:41.039 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.7 MB/s | 970 kB 00:00 2024-05-21T12:14:41.052 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:41.075 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:41.075 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:41.158 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-fuse 2024-05-21T12:14:41.159 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:41.159 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:14:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-21T12:14:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/2 2024-05-21T12:14:41.684 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:08 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:41.793 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:41.793 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:41.793 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:41.793 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:41.793 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:41.794 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:41.805 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 890 k 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:41.806 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:41.807 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:41.807 INFO:teuthology.orchestra.run.smithi072.stdout:Install 2 Packages 2024-05-21T12:14:41.807 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:41.807 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 970 k 2024-05-21T12:14:41.807 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 2.8 M 2024-05-21T12:14:41.807 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:42.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-nbd 2024-05-21T12:14:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/2 2024-05-21T12:14:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/2 2024-05-21T12:14:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-21T12:14:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:42.533 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:14:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-21T12:14:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:42.533 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:42.557 INFO:teuthology.orchestra.run.smithi072.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 141 kB/s | 80 kB 00:00 2024-05-21T12:14:42.615 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:56 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:42.742 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:42.744 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-05-21T12:14:42.744 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:42.744 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-21T12:14:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 171 k 2024-05-21T12:14:42.744 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:42.744 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:42.744 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:42.745 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-05-21T12:14:42.745 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:42.745 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 171 k 2024-05-21T12:14:42.745 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 494 k 2024-05-21T12:14:42.745 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:42.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-volume 2024-05-21T12:14:43.232 INFO:teuthology.orchestra.run.smithi072.stdout:(2/2): ceph-fuse-19.0.0-3665.gb0c764b6.el9.x86_ 716 kB/s | 890 kB 00:01 2024-05-21T12:14:43.232 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:43.233 INFO:teuthology.orchestra.run.smithi072.stdout:Total 680 kB/s | 970 kB 00:01 2024-05-21T12:14:43.247 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:43.268 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-nbd-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 326 kB/s | 171 kB 00:00 2024-05-21T12:14:43.269 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:43.269 INFO:teuthology.orchestra.run.smithi092.stdout:Total 325 kB/s | 171 kB 00:00 2024-05-21T12:14:43.269 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:43.271 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:43.271 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:43.277 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:43.278 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:43.343 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:43.343 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:43.351 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:03 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:43.352 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:43.352 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:43.479 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:43.479 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:43.479 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-21T12:14:43.479 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:43.479 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:43.480 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 273 k 2024-05-21T12:14:43.480 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-21T12:14:43.480 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T12:14:43.480 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:43.480 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:43.480 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:43.480 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2024-05-21T12:14:43.480 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:43.481 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 350 k 2024-05-21T12:14:43.481 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.5 M 2024-05-21T12:14:43.481 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:43.503 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:43.540 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:43.687 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 557 kB/s | 77 kB 00:00 2024-05-21T12:14:43.692 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-21T12:14:43.774 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/2 2024-05-21T12:14:43.978 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): ceph-volume-19.0.0-3665.gb0c764b6.el9.no 634 kB/s | 273 kB 00:00 2024-05-21T12:14:43.979 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:43.979 INFO:teuthology.orchestra.run.smithi045.stdout:Total 702 kB/s | 350 kB 00:00 2024-05-21T12:14:43.991 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:44.002 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:44.002 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:44.038 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:44.039 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:44.185 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:14:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:44.267 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-21T12:14:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:44.267 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:44.268 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:44.379 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-21T12:14:44.406 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2/2 2024-05-21T12:14:44.423 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2/2 2024-05-21T12:14:44.423 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:14:44.423 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T12:14:44.423 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:44.481 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2/2 2024-05-21T12:14:44.481 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/2 2024-05-21T12:14:44.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install nvme-cli 2024-05-21T12:14:44.763 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-21T12:14:44.763 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:44.763 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:44.764 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-21T12:14:44.764 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:44.764 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:45.039 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:59 ago on Tue 21 May 2024 12:12:46 PM UTC. 2024-05-21T12:14:45.086 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-volume 2024-05-21T12:14:45.098 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 1/2 2024-05-21T12:14:45.120 INFO:teuthology.orchestra.run.smithi092.stdout:Package nvme-cli-2.6-5.el9.x86_64 is already installed. 2024-05-21T12:14:45.163 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-21T12:14:45.163 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: libnvme x86_64 1.9-1.el9 baseos 104 k 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: nvme-cli x86_64 2.9.1-2.el9 baseos 906 k 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-05-21T12:14:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 1.0 M 2024-05-21T12:14:45.165 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-21T12:14:45.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-21T12:14:45.407 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:45.407 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:14:45.407 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:45.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T12:14:45.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:45.408 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:45.633 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:12 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:45.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados-devel 2024-05-21T12:14:45.757 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:19.0.0-3665.gb0c764b6.el9 ceph-noarch 273 k 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:45.758 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:45.759 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:45.759 INFO:teuthology.orchestra.run.smithi072.stdout:Install 2 Packages 2024-05-21T12:14:45.759 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:45.759 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 350 k 2024-05-21T12:14:45.759 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 1.5 M 2024-05-21T12:14:45.759 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:45.782 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): libnvme-1.9-1.el9.x86_64.rpm 231 kB/s | 104 kB 00:00 2024-05-21T12:14:45.962 INFO:teuthology.orchestra.run.smithi072.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 568 kB/s | 77 kB 00:00 2024-05-21T12:14:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): nvme-cli-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 906 kB 00:00 2024-05-21T12:14:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.0 MB/s | 1.0 MB 00:00 2024-05-21T12:14:46.137 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-21T12:14:46.149 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-21T12:14:46.149 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-21T12:14:46.171 INFO:teuthology.orchestra.run.smithi072.stdout:(2/2): ceph-volume-19.0.0-3665.gb0c764b6.el9.no 792 kB/s | 273 kB 00:00 2024-05-21T12:14:46.171 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:46.172 INFO:teuthology.orchestra.run.smithi072.stdout:Total 847 kB/s | 350 kB 00:00 2024-05-21T12:14:46.184 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:46.193 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:46.193 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:46.226 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:46.226 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:46.267 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-21T12:14:46.267 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-21T12:14:46.280 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:06 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:46.368 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:46.406 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:46.407 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:46.407 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-05-21T12:14:46.407 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:46.407 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:46.407 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 127 k 2024-05-21T12:14:46.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:46.408 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:46.408 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:46.408 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-05-21T12:14:46.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:46.408 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 127 k 2024-05-21T12:14:46.408 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 456 k 2024-05-21T12:14:46.408 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:46.514 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-21T12:14:46.560 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-21T12:14:46.603 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2/2 2024-05-21T12:14:46.619 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2/2 2024-05-21T12:14:46.619 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:14:46.619 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T12:14:46.619 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:46.620 INFO:teuthology.orchestra.run.smithi045.stdout:librados-devel-19.0.0-3665.gb0c764b6.el9.x86_64 601 kB/s | 127 kB 00:00 2024-05-21T12:14:46.620 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:46.621 INFO:teuthology.orchestra.run.smithi045.stdout:Total 597 kB/s | 127 kB 00:00 2024-05-21T12:14:46.621 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:46.626 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:46.626 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:46.693 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:46.694 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:46.835 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libnvme-1.9-1.el9.x86_64 1/4 2024-05-21T12:14:46.850 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:14:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : nvme-cli-2.9.1-2.el9.x86_64 2/4 2024-05-21T12:14:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:47.207 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: nvme-cli-2.9.1-2.el9.x86_64 2/4 2024-05-21T12:14:47.249 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : nvme-cli-2.6-5.el9.x86_64 3/4 2024-05-21T12:14:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libnvme-1.6-1.el9.x86_64 4/4 2024-05-21T12:14:47.354 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:47.400 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 1/2 2024-05-21T12:14:47.605 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:47.605 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:47.605 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:14:47.605 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:47.605 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:47.606 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:47.715 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-21T12:14:47.716 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:47.716 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:47.716 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:19.0.0-3665.gb0c764b6.el9.noarch 2024-05-21T12:14:47.716 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T12:14:47.716 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:47.716 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:47.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs2 2024-05-21T12:14:48.096 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librados-devel 2024-05-21T12:14:48.139 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libnvme-1.6-1.el9.x86_64 4/4 2024-05-21T12:14:48.139 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libnvme-1.9-1.el9.x86_64 1/4 2024-05-21T12:14:48.140 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libnvme-1.6-1.el9.x86_64 2/4 2024-05-21T12:14:48.140 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : nvme-cli-2.9.1-2.el9.x86_64 3/4 2024-05-21T12:14:48.409 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:08 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : nvme-cli-2.6-5.el9.x86_64 4/4 2024-05-21T12:14:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:48.459 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-05-21T12:14:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: libnvme-1.9-1.el9.x86_64 nvme-cli-2.9.1-2.el9.x86_64 2024-05-21T12:14:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:14:48.459 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-21T12:14:48.493 INFO:teuthology.orchestra.run.smithi045.stdout:Package libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:48.537 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:48.538 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:14:48.538 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:48.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs-devel 2024-05-21T12:14:48.636 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:15 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:48.760 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 127 k 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:48.761 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 127 k 2024-05-21T12:14:48.762 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 456 k 2024-05-21T12:14:48.762 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:48.791 DEBUG:teuthology.parallel:result is None 2024-05-21T12:14:49.171 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:09 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:49.298 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 32 k 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-05-21T12:14:49.299 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:49.300 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 32 k 2024-05-21T12:14:49.300 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 142 k 2024-05-21T12:14:49.300 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:49.401 INFO:teuthology.orchestra.run.smithi072.stdout:librados-devel-19.0.0-3665.gb0c764b6.el9.x86_64 198 kB/s | 127 kB 00:00 2024-05-21T12:14:49.402 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:49.402 INFO:teuthology.orchestra.run.smithi072.stdout:Total 197 kB/s | 127 kB 00:00 2024-05-21T12:14:49.402 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:49.408 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:49.408 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:49.473 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:49.474 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:49.626 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:49.707 INFO:teuthology.orchestra.run.smithi045.stdout:libcephfs-devel-19.0.0-3665.gb0c764b6.el9.x86_6 78 kB/s | 32 kB 00:00 2024-05-21T12:14:49.708 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:49.708 INFO:teuthology.orchestra.run.smithi045.stdout:Total 78 kB/s | 32 kB 00:00 2024-05-21T12:14:49.708 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:49.712 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:49.712 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:49.729 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:49.735 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:49.735 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:49.818 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:14:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:50.166 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:50.423 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:50.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:50.424 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:50.424 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:50.424 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:50.424 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:50.505 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:50.505 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:50.506 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:14:50.506 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:50.506 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:50.506 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:50.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados2 2024-05-21T12:14:50.719 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install libcephfs2 2024-05-21T12:14:51.236 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:11 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:51.250 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:18 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:51.319 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:51.329 INFO:teuthology.orchestra.run.smithi072.stdout:Package libcephfs2-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:51.365 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:51.366 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:14:51.366 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:51.371 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:51.372 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:14:51.372 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:51.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librbd1 2024-05-21T12:14:51.467 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install libcephfs-devel 2024-05-21T12:14:51.995 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:11 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:51.996 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:18 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:52.078 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:52.116 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 32 k 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:14:52.117 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-05-21T12:14:52.118 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:52.118 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 32 k 2024-05-21T12:14:52.118 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 142 k 2024-05-21T12:14:52.118 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:14:52.122 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:52.123 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:14:52.123 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:52.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rados 2024-05-21T12:14:52.355 INFO:teuthology.orchestra.run.smithi072.stdout:libcephfs-devel-19.0.0-3665.gb0c764b6.el9.x86_6 134 kB/s | 32 kB 00:00 2024-05-21T12:14:52.356 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:52.356 INFO:teuthology.orchestra.run.smithi072.stdout:Total 133 kB/s | 32 kB 00:00 2024-05-21T12:14:52.356 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:14:52.359 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:14:52.360 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:14:52.380 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:14:52.381 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:14:52.464 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:14:52.572 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:52.762 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:12 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:52.844 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:52.888 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:52.889 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:14:52.890 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:52.977 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rgw 2024-05-21T12:14:52.980 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:53.186 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:53.186 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:53.186 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:14:53.186 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:53.186 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:14:53.186 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:53.533 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:13 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:53.616 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:53.660 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:53.661 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:14:53.661 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:53.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-cephfs 2024-05-21T12:14:54.297 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:14 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:54.367 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librados2 2024-05-21T12:14:54.381 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:54.425 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:54.426 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:14:54.426 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:54.510 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rbd 2024-05-21T12:14:54.908 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:21 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:54.990 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:55.032 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:55.033 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:14:55.033 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:55.066 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:15 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:55.150 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:55.194 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:55.195 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-05-21T12:14:55.195 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:55.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-fuse 2024-05-21T12:14:55.524 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librbd1 2024-05-21T12:14:55.827 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:15 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:55.954 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:55.955 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-05-21T12:14:55.955 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:55.955 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 85 k 2024-05-21T12:14:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:55.955 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:55.955 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:55.956 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-05-21T12:14:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:55.956 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 85 k 2024-05-21T12:14:55.956 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 231 k 2024-05-21T12:14:55.956 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:56.052 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:23 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:56.131 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:56.162 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-fuse-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 415 kB/s | 85 kB 00:00 2024-05-21T12:14:56.162 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:56.162 INFO:teuthology.orchestra.run.smithi045.stdout:Total 412 kB/s | 85 kB 00:00 2024-05-21T12:14:56.163 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:56.170 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:56.170 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:56.172 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:56.173 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:14:56.173 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:56.238 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:56.238 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:56.385 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:14:56.496 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:56.656 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rados 2024-05-21T12:14:56.865 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:57.103 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:14:57.103 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:57.103 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:14:57.103 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:14:57.103 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:57.103 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:14:57.187 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:24 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:57.267 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rados-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:57.309 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:57.310 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:14:57.310 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:57.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-mirror 2024-05-21T12:14:57.762 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rgw 2024-05-21T12:14:57.926 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:17 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:14:58.054 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:14:58.055 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:58.055 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-05-21T12:14:58.055 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:58.055 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:14:58.055 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.1 M 2024-05-21T12:14:58.055 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:58.055 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:14:58.056 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:14:58.056 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-05-21T12:14:58.056 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:14:58.056 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 3.1 M 2024-05-21T12:14:58.056 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 13 M 2024-05-21T12:14:58.056 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:14:58.288 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:25 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:58.367 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rgw-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:58.408 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:58.409 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:14:58.409 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:58.486 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-cephfs 2024-05-21T12:14:59.020 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:26 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:59.099 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-cephfs-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:59.140 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:59.142 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:14:59.142 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:59.219 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rbd 2024-05-21T12:14:59.565 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-mirror-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 2.0 MB/s | 3.1 MB 00:01 2024-05-21T12:14:59.566 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:14:59.566 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.0 MB/s | 3.1 MB 00:01 2024-05-21T12:14:59.566 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:14:59.577 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:14:59.577 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:14:59.663 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:14:59.663 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:14:59.744 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:26 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:14:59.823 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 is already installed. 2024-05-21T12:14:59.865 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:14:59.866 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-05-21T12:14:59.866 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:14:59.935 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-fuse 2024-05-21T12:15:00.116 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:15:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:00.210 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:15:00.210 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T12:15:00.210 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T12:15:00.210 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T12:15:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:00.462 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:27 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:15:00.585 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 85 k 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:00.586 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-05-21T12:15:00.587 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:00.587 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 85 k 2024-05-21T12:15:00.587 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 231 k 2024-05-21T12:15:00.587 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:15:00.975 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-fuse-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 219 kB/s | 85 kB 00:00 2024-05-21T12:15:00.976 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:15:00.976 INFO:teuthology.orchestra.run.smithi072.stdout:Total 219 kB/s | 85 kB 00:00 2024-05-21T12:15:00.976 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:15:00.983 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:15:00.983 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:15:01.047 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:15:01.047 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:15:01.195 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:15:01.293 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:01.414 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:01.414 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:01.414 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:15:01.414 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:15:01.415 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:01.415 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:15:01.646 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:01.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-nbd 2024-05-21T12:15:01.892 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:01.892 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:01.892 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:15:01.892 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:15:01.892 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:01.892 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:15:02.158 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-mirror 2024-05-21T12:15:02.278 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:22 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:15:02.405 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:15:02.406 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:15:02.406 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-05-21T12:15:02.406 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 171 k 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 171 k 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 494 k 2024-05-21T12:15:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:15:02.699 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:29 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:15:02.806 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-nbd-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 428 kB/s | 171 kB 00:00 2024-05-21T12:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout:Total 427 kB/s | 171 kB 00:00 2024-05-21T12:15:02.807 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:15:02.816 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:15:02.822 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 3.1 M 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:02.823 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-05-21T12:15:02.824 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:02.824 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 3.1 M 2024-05-21T12:15:02.824 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 13 M 2024-05-21T12:15:02.824 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:15:02.882 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:15:02.883 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:15:03.043 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:15:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:03.561 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:03.837 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:03.837 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:03.837 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-21T12:15:03.837 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:15:03.837 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:03.837 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:15:04.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install nvme-cli 2024-05-21T12:15:04.408 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-mirror-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 2.0 MB/s | 3.1 MB 00:01 2024-05-21T12:15:04.408 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:15:04.409 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.9 MB/s | 3.1 MB 00:01 2024-05-21T12:15:04.409 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:15:04.420 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:15:04.420 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:15:04.505 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:15:04.505 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:15:04.668 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:24 ago on Tue 21 May 2024 12:12:40 PM UTC. 2024-05-21T12:15:04.750 INFO:teuthology.orchestra.run.smithi045.stdout:Package nvme-cli-2.6-5.el9.x86_64 is already installed. 2024-05-21T12:15:04.796 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-21T12:15:04.797 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:15:04.798 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-05-21T12:15:04.798 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:15:04.798 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-05-21T12:15:04.798 INFO:teuthology.orchestra.run.smithi045.stdout: libnvme x86_64 1.9-1.el9 baseos 104 k 2024-05-21T12:15:04.798 INFO:teuthology.orchestra.run.smithi045.stdout: nvme-cli x86_64 2.9.1-2.el9 baseos 906 k 2024-05-21T12:15:04.798 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:04.798 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-21T12:15:04.799 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-21T12:15:04.799 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 2 Packages 2024-05-21T12:15:04.799 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:04.799 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 1.0 M 2024-05-21T12:15:04.799 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-21T12:15:04.930 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:15:04.992 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:05.017 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:05.017 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T12:15:05.018 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T12:15:05.018 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T12:15:05.018 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T12:15:05.018 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:05.838 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): libnvme-1.9-1.el9.x86_64.rpm 136 kB/s | 104 kB 00:00 2024-05-21T12:15:06.054 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): nvme-cli-2.9.1-2.el9.x86_64.rpm 918 kB/s | 906 kB 00:00 2024-05-21T12:15:06.055 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:15:06.055 INFO:teuthology.orchestra.run.smithi045.stdout:Total 803 kB/s | 1.0 MB 00:01 2024-05-21T12:15:06.088 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-21T12:15:06.101 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-21T12:15:06.101 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-21T12:15:06.207 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:06.207 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:06.207 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:15:06.207 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:15:06.207 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:06.207 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:15:06.218 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-21T12:15:06.218 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-21T12:15:06.471 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-21T12:15:06.487 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-nbd 2024-05-21T12:15:06.782 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libnvme-1.9-1.el9.x86_64 1/4 2024-05-21T12:15:06.844 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : nvme-cli-2.9.1-2.el9.x86_64 2/4 2024-05-21T12:15:07.030 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:34 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:15:07.161 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:19.0.0-3665.gb0c764b6.el9 ceph 171 k 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-05-21T12:15:07.162 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:07.163 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 171 k 2024-05-21T12:15:07.163 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 494 k 2024-05-21T12:15:07.163 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:15:07.172 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: nvme-cli-2.9.1-2.el9.x86_64 2/4 2024-05-21T12:15:07.206 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : nvme-cli-2.6-5.el9.x86_64 3/4 2024-05-21T12:15:07.313 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libnvme-1.6-1.el9.x86_64 4/4 2024-05-21T12:15:07.769 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-nbd-19.0.0-3665.gb0c764b6.el9.x86_64.rpm 282 kB/s | 171 kB 00:00 2024-05-21T12:15:07.770 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:15:07.771 INFO:teuthology.orchestra.run.smithi072.stdout:Total 281 kB/s | 171 kB 00:00 2024-05-21T12:15:07.771 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:15:07.779 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:15:07.779 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:15:07.845 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:15:07.846 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:15:07.999 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:15:08.073 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libnvme-1.6-1.el9.x86_64 4/4 2024-05-21T12:15:08.073 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libnvme-1.9-1.el9.x86_64 1/4 2024-05-21T12:15:08.073 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libnvme-1.6-1.el9.x86_64 2/4 2024-05-21T12:15:08.073 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : nvme-cli-2.9.1-2.el9.x86_64 3/4 2024-05-21T12:15:08.103 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:08.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : nvme-cli-2.6-5.el9.x86_64 4/4 2024-05-21T12:15:08.361 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:08.361 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-05-21T12:15:08.361 INFO:teuthology.orchestra.run.smithi045.stdout: libnvme-1.9-1.el9.x86_64 nvme-cli-2.9.1-2.el9.x86_64 2024-05-21T12:15:08.361 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:15:08.362 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-21T12:15:08.549 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:08.708 DEBUG:teuthology.parallel:result is None 2024-05-21T12:15:08.824 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 1/1 2024-05-21T12:15:08.825 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:08.825 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-21T12:15:08.825 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:19.0.0-3665.gb0c764b6.el9.x86_64 2024-05-21T12:15:08.825 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:08.825 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:15:09.094 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install nvme-cli 2024-05-21T12:15:09.640 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:36 ago on Tue 21 May 2024 12:12:33 PM UTC. 2024-05-21T12:15:09.722 INFO:teuthology.orchestra.run.smithi072.stdout:Package nvme-cli-2.6-5.el9.x86_64 is already installed. 2024-05-21T12:15:09.765 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-21T12:15:09.765 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:09.765 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: libnvme x86_64 1.9-1.el9 baseos 104 k 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: nvme-cli x86_64 2.9.1-2.el9 baseos 906 k 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 2 Packages 2024-05-21T12:15:09.766 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:09.767 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 1.0 M 2024-05-21T12:15:09.767 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-21T12:15:10.002 INFO:teuthology.orchestra.run.smithi072.stdout:(1/2): libnvme-1.9-1.el9.x86_64.rpm 554 kB/s | 104 kB 00:00 2024-05-21T12:15:10.352 INFO:teuthology.orchestra.run.smithi072.stdout:(2/2): nvme-cli-2.9.1-2.el9.x86_64.rpm 1.6 MB/s | 906 kB 00:00 2024-05-21T12:15:10.353 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:15:10.353 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.7 MB/s | 1.0 MB 00:00 2024-05-21T12:15:10.381 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-21T12:15:10.392 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-21T12:15:10.392 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-21T12:15:10.492 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-21T12:15:10.492 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-21T12:15:10.738 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-21T12:15:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libnvme-1.9-1.el9.x86_64 1/4 2024-05-21T12:15:11.258 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : nvme-cli-2.9.1-2.el9.x86_64 2/4 2024-05-21T12:15:11.544 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: nvme-cli-2.9.1-2.el9.x86_64 2/4 2024-05-21T12:15:11.587 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : nvme-cli-2.6-5.el9.x86_64 3/4 2024-05-21T12:15:11.684 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libnvme-1.6-1.el9.x86_64 4/4 2024-05-21T12:15:12.571 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libnvme-1.6-1.el9.x86_64 4/4 2024-05-21T12:15:12.571 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libnvme-1.9-1.el9.x86_64 1/4 2024-05-21T12:15:12.571 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libnvme-1.6-1.el9.x86_64 2/4 2024-05-21T12:15:12.571 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : nvme-cli-2.9.1-2.el9.x86_64 3/4 2024-05-21T12:15:12.854 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : nvme-cli-2.6-5.el9.x86_64 4/4 2024-05-21T12:15:12.855 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:12.855 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-05-21T12:15:12.855 INFO:teuthology.orchestra.run.smithi072.stdout: libnvme-1.9-1.el9.x86_64 nvme-cli-2.9.1-2.el9.x86_64 2024-05-21T12:15:12.855 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:15:12.855 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-21T12:15:13.290 DEBUG:teuthology.parallel:result is None 2024-05-21T12:15:13.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:15:13.440 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T12:15:13.468 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-3665.gb0c764b6.el9 2024-05-21T12:15:13.469 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3665.gb0c764b6.el9 2024-05-21T12:15:13.469 INFO:teuthology.task.install:The correct ceph version 19.0.0-3665.gb0c764b6 is installed. 2024-05-21T12:15:13.471 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:15:13.613 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T12:15:13.635 INFO:teuthology.orchestra.run.smithi072.stdout:19.0.0-3665.gb0c764b6.el9 2024-05-21T12:15:13.635 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3665.gb0c764b6.el9 2024-05-21T12:15:13.635 INFO:teuthology.task.install:The correct ceph version 19.0.0-3665.gb0c764b6 is installed. 2024-05-21T12:15:13.638 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:15:13.779 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T12:15:13.806 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-3665.gb0c764b6.el9 2024-05-21T12:15:13.806 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3665.gb0c764b6.el9 2024-05-21T12:15:13.806 INFO:teuthology.task.install:The correct ceph version 19.0.0-3665.gb0c764b6 is installed. 2024-05-21T12:15:13.808 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:15:14.027 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T12:15:14.053 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-3665.gb0c764b6.el9 2024-05-21T12:15:14.054 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3665.gb0c764b6.el9 2024-05-21T12:15:14.054 INFO:teuthology.task.install:The correct ceph version 19.0.0-3665.gb0c764b6 is installed. 2024-05-21T12:15:14.056 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-21T12:15:14.056 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:15:14.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T12:15:14.090 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:15:14.090 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T12:15:14.122 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:15:14.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T12:15:14.152 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:15:14.153 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T12:15:14.185 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-21T12:15:14.185 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:15:14.185 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T12:15:14.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T12:15:14.286 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:15:14.286 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T12:15:14.311 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T12:15:14.377 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:15:14.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T12:15:14.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T12:15:14.477 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:15:14.477 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T12:15:14.507 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T12:15:14.573 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-21T12:15:14.574 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:15:14.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T12:15:14.604 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T12:15:14.675 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:15:14.675 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T12:15:14.701 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T12:15:14.767 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:15:14.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T12:15:14.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T12:15:14.861 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:15:14.861 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T12:15:14.887 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T12:15:14.952 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-21T12:15:14.953 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:15:14.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T12:15:14.983 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T12:15:15.054 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:15:15.054 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T12:15:15.080 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T12:15:15.146 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:15:15.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T12:15:15.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T12:15:15.235 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:15:15.236 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T12:15:15.261 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T12:15:15.327 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-21T12:15:15.428 INFO:tasks.cephadm:Config: {'watchdog_setup': None, 'conf': {'global': {'mon warn on pool no app': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'debug ms': 1, 'debug osd': 20}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_FAILED_DAEMON', 'is in error state', 'failed cephadm daemon'], 'sha1': 'b0c764b6e22fbb51f59e469d9dd99895c152f73e'} 2024-05-21T12:15:15.428 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:15:15.428 INFO:tasks.cephadm:Cluster fsid is c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:15:15.428 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-21T12:15:15.429 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.72', 'mon.c': '172.21.15.92'} 2024-05-21T12:15:15.430 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-05-21T12:15:15.430 INFO:tasks.cephadm:First mgr is x 2024-05-21T12:15:15.430 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-21T12:15:15.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-05-21T12:15:15.462 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname $(hostname -s) 2024-05-21T12:15:15.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-05-21T12:15:15.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-05-21T12:15:15.540 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-21T12:15:15.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:15:15.765 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b0c764b6e22fbb51f59e469d9dd99895c152f73e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79441/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3665-gb0c764b6', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3665.gb0c764b6'}, 'url': 'https://2.chacra.ceph.com/r/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-16 09:38:55.309783', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'ceph-nvmeof-mon', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-21T12:15:15.904 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref ceph-nvmeof-mon, sha1 b0c764b6e22fbb51f59e469d9dd99895c152f73e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:15:15.905 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/x86_64/flavors/default/cephadm 2024-05-21T12:15:15.905 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:16.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 782984 May 21 12:15 /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:16.926 DEBUG:teuthology.orchestra.run.smithi072:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:18.071 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r--. 1 ubuntu ubuntu 782984 May 21 12:15 /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:18.071 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:19.277 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 782984 May 21 12:15 /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:19.277 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/ceph-nvmeof-mon/b0c764b6e22fbb51f59e469d9dd99895c152f73e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:20.266 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 782984 May 21 12:15 /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:20.267 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:20.287 DEBUG:teuthology.orchestra.run.smithi072:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:20.308 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:20.327 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T12:15:20.361 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e on all hosts... 2024-05-21T12:15:20.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e pull 2024-05-21T12:15:20.364 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e pull 2024-05-21T12:15:20.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e pull 2024-05-21T12:15:20.369 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e pull 2024-05-21T12:15:21.170 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e... 2024-05-21T12:15:21.207 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e... 2024-05-21T12:15:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e... 2024-05-21T12:15:21.265 INFO:teuthology.orchestra.run.smithi072.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e... 2024-05-21T12:15:56.886 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-05-21T12:15:56.887 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)", 2024-05-21T12:15:56.887 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815", 2024-05-21T12:15:56.887 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-05-21T12:15:56.887 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93" 2024-05-21T12:15:56.887 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-05-21T12:15:56.887 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-05-21T12:15:58.169 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-21T12:15:58.169 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)", 2024-05-21T12:15:58.170 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815", 2024-05-21T12:15:58.170 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-05-21T12:15:58.170 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93" 2024-05-21T12:15:58.170 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-21T12:15:58.170 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-21T12:16:17.270 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-21T12:16:17.271 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)", 2024-05-21T12:16:17.271 INFO:teuthology.orchestra.run.smithi072.stdout: "image_id": "1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815", 2024-05-21T12:16:17.271 INFO:teuthology.orchestra.run.smithi072.stdout: "repo_digests": [ 2024-05-21T12:16:17.271 INFO:teuthology.orchestra.run.smithi072.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93" 2024-05-21T12:16:17.271 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-05-21T12:16:17.271 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-21T12:16:19.947 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-21T12:16:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)", 2024-05-21T12:16:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815", 2024-05-21T12:16:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-05-21T12:16:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93" 2024-05-21T12:16:19.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-05-21T12:16:19.947 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-21T12:16:19.973 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-05-21T12:16:20.008 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph 2024-05-21T12:16:20.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-05-21T12:16:20.071 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-05-21T12:16:20.104 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-05-21T12:16:20.135 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /etc/ceph 2024-05-21T12:16:20.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-05-21T12:16:20.190 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-05-21T12:16:20.219 INFO:tasks.cephadm:Writing seed config... 2024-05-21T12:16:20.220 INFO:tasks.cephadm: override: [global] mon warn on pool no app = False 2024-05-21T12:16:20.220 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-21T12:16:20.220 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-21T12:16:20.220 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-21T12:16:20.221 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-21T12:16:20.221 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-21T12:16:20.221 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-21T12:16:20.221 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-21T12:16:20.221 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-21T12:16:20.221 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:16:20.222 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-21T12:16:20.240 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c7e3752a-176b-11ef-bc9a-c7b262605968 mon warn on pool no app = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-05-21T12:16:20.241 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.a.service 2024-05-21T12:16:20.283 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-21T12:16:20.326 INFO:tasks.cephadm:Bootstrapping... 2024-05-21T12:16:20.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e -v bootstrap --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:16:20.527 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:16:20.527 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e', '-v', 'bootstrap', '--fsid', 'c7e3752a-176b-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2024-05-21T12:16:20.528 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-21T12:16:20.528 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-05-21T12:16:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 5.0.2 2024-05-21T12:16:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-05-21T12:16:20.555 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-05-21T12:16:20.563 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T12:16:20.564 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T12:16:20.570 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T12:16:20.570 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-05-21T12:16:20.578 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-05-21T12:16:20.586 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-05-21T12:16:20.586 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-05-21T12:16:20.586 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-05-21T12:16:20.612 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 5.0.2 2024-05-21T12:16:20.612 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-21T12:16:20.612 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-05-21T12:16:20.612 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-05-21T12:16:20.619 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T12:16:20.619 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T12:16:20.626 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T12:16:20.626 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-05-21T12:16:20.633 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-05-21T12:16:20.641 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-05-21T12:16:20.641 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-05-21T12:16:20.641 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-05-21T12:16:20.641 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:20.642 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 140101046899616 on /run/cephadm/c7e3752a-176b-11ef-bc9a-c7b262605968.lock 2024-05-21T12:16:20.642 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 140101046899616 acquired on /run/cephadm/c7e3752a-176b-11ef-bc9a-c7b262605968.lock 2024-05-21T12:16:20.642 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2024-05-21T12:16:20.643 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-05-21T12:16:20.643 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-05-21T12:16:20.647 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-05-21T12:16:20.647 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-05-21T12:16:20.650 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-21T12:16:20.650 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-21T12:16:20.650 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-21T12:16:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-21T12:16:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-21T12:16:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T12:16:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-21T12:16:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-21T12:16:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T12:16:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-05-21T12:16:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-05-21T12:16:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-21T12:16:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-21T12:16:20.655 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e... 2024-05-21T12:16:23.609 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 2024-05-21T12:16:23.609 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e... 2024-05-21T12:16:23.609 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-05-21T12:16:23.609 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-21T12:16:23.610 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:71d51d8882484ab8e8af8ea921074e1ad6716383a3f405b7394cee80a640ea30 2024-05-21T12:16:23.610 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 2024-05-21T12:16:23.610 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-21T12:16:25.518 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:16:25.519 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:16:25.519 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-05-21T12:16:26.467 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-05-21T12:16:26.467 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-05-21T12:16:27.272 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCakExmEqjnKxAAPUOmYFeyE8W7T957lFHCAg== 2024-05-21T12:16:28.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCbkExm07cXJBAAcgZSU7L53kNHVtK1ifhhQg== 2024-05-21T12:16:29.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCckExmP7CoHBAAqszWwqhZtAnrRyxNjoA/MQ== 2024-05-21T12:16:29.068 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-05-21T12:16:29.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T12:16:29.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-21T12:16:29.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:29.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T12:16:29.949 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T12:16:29.949 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-05-21T12:16:29.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:29.949 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T12:16:29.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:16:29.950 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-05-21T12:16:31.429 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-05-21T12:16:31.653 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-21T12:16:31.797 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-21T12:16:31.982 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968.target. 2024-05-21T12:16:31.982 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968.target. 2024-05-21T12:16:32.243 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.a 2024-05-21T12:16:32.244 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-21T12:16:32.421 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968.target.wants/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service. 2024-05-21T12:16:32.981 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T12:16:32.982 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-05-21T12:16:32.989 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T12:16:32.989 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-05-21T12:16:32.989 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-05-21T12:16:32.989 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T12:16:32.990 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-05-21T12:16:32.990 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-05-21T12:16:33.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:32 smithi045 systemd[1]: Started Ceph mon.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:16:34.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T12:16:34.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: monmap epoch 1 2024-05-21T12:16:34.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:34.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: last_changed 2024-05-21T12:16:29.401381+0000 2024-05-21T12:16:34.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: created 2024-05-21T12:16:29.401381+0000 2024-05-21T12:16:34.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: min_mon_release 19 (squid) 2024-05-21T12:16:34.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: election_strategy: 1 2024-05-21T12:16:34.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-05-21T12:16:34.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: fsmap 2024-05-21T12:16:34.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: osdmap e1: 0 total, 0 up, 0 in 2024-05-21T12:16:34.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: mgrmap e1: no daemons active 2024-05-21T12:16:34.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:33 smithi045 ceph-mon[29790]: from='client.? 172.21.15.45:0/3428753534' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:16:34.051 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-05-21T12:16:34.051 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:34.051 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-21T12:16:34.051 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.55219s) 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-05-21T12:16:34.052 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-05-21T12:16:35.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:34 smithi045 ceph-mon[29790]: from='client.? 172.21.15.45:0/3790429720' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-05-21T12:16:35.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:34 smithi045 ceph-mon[29790]: from='client.? 172.21.15.45:0/3790429720' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-21T12:16:35.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T12:16:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-21T12:16:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-05-21T12:16:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T12:16:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T12:16:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T12:16:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T12:16:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T12:16:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T12:16:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T12:16:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T12:16:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T12:16:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-05-21T12:16:36.147 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-05-21T12:16:36.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:35 smithi045 ceph-mon[29790]: from='client.? 172.21.15.45:0/2468254613' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:16:36.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:36 smithi045 systemd[1]: Stopping Ceph mon.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:16:36.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a[29786]: 2024-05-21T12:16:36.405+0000 7f019119d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:16:36.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a[29786]: 2024-05-21T12:16:36.405+0000 7f019119d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-21T12:16:36.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:36 smithi045 podman[30038]: 2024-05-21 12:16:36.630767119 +0000 UTC m=+0.317898516 container died 1fbab912895b8c64e75f3df2d30d2f3622ada33ab3ac0f5c0e738cee8613ee6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=ceph-nvmeof-mon-b0c764b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-21T12:16:37.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:36 smithi045 podman[30038]: 2024-05-21 12:16:36.971433843 +0000 UTC m=+0.658565225 container cleanup 1fbab912895b8c64e75f3df2d30d2f3622ada33ab3ac0f5c0e738cee8613ee6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T12:16:37.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:36 smithi045 bash[30038]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a 2024-05-21T12:16:37.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 podman[30050]: 2024-05-21 12:16:37.155709924 +0000 UTC m=+0.522121403 container remove 1fbab912895b8c64e75f3df2d30d2f3622ada33ab3ac0f5c0e738cee8613ee6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T12:16:37.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-21T12:16:37.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 systemd[1]: Stopped Ceph mon.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:16:37.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 systemd[1]: Starting Ceph mon.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:16:37.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 podman[30144]: 2024-05-21 12:16:37.521293342 +0000 UTC m=+0.017861404 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e 2024-05-21T12:16:37.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 podman[30144]: 2024-05-21 12:16:37.623387271 +0000 UTC m=+0.119955332 container create 80d88adb7cd3fb8ffc9a79f419c020917186deff3a8ad2862cff51735a3090b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-21T12:16:37.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 podman[30144]: 2024-05-21 12:16:37.859077292 +0000 UTC m=+0.355645329 container init 80d88adb7cd3fb8ffc9a79f419c020917186deff3a8ad2862cff51735a3090b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-05-21T12:16:37.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 podman[30144]: 2024-05-21 12:16:37.862349671 +0000 UTC m=+0.358917714 container start 80d88adb7cd3fb8ffc9a79f419c020917186deff3a8ad2862cff51735a3090b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-21T12:16:37.963 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-21T12:16:38.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: set uid:gid to 167:167 (ceph:ceph) 2024-05-21T12:16:38.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev), process ceph-mon, pid 2 2024-05-21T12:16:38.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: pidfile_write: ignore empty --pid-file 2024-05-21T12:16:38.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: load: jerasure load: lrc 2024-05-21T12:16:38.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: RocksDB version: 7.9.2 2024-05-21T12:16:38.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Git sha 0 2024-05-21T12:16:38.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Compile date 2024-05-16 09:00:48 2024-05-21T12:16:38.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: DB SUMMARY 2024-05-21T12:16:38.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: DB Session ID: 7WCNIY7NL15O9VY4Z919 2024-05-21T12:16:38.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: CURRENT file: CURRENT 2024-05-21T12:16:38.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T12:16:38.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-21T12:16:38.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-21T12:16:38.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 89238 ; 2024-05-21T12:16:38.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.error_if_exists: 0 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.create_if_missing: 0 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.paranoid_checks: 1 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.env: 0x55625b0b30c0 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.info_log: 0x55625d4adde0 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.statistics: (nil) 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.use_fsync: 0 2024-05-21T12:16:38.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_log_file_size: 0 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.allow_fallocate: 1 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.use_direct_reads: 0 2024-05-21T12:16:38.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T12:16:38.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T12:16:38.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.db_log_dir: 2024-05-21T12:16:38.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.wal_dir: 2024-05-21T12:16:38.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T12:16:38.186 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.write_buffer_manager: 0x55625d47e1e0 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T12:16:38.187 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.unordered_write: 0 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.row_cache: None 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.wal_filter: None 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T12:16:38.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.two_write_queues: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.wal_compression: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.atomic_flush: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.log_readahead_size: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T12:16:38.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_background_jobs: 2 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_background_compactions: -1 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_subcompactions: 1 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T12:16:38.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_open_files: -1 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_background_flushes: -1 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Compression algorithms supported: 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: kZSTD supported: 0 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: kXpressCompression supported: 0 2024-05-21T12:16:38.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: kZlibCompression supported: 1 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: kSnappyCompression supported: 1 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: kLZ4Compression supported: 1 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: kBZip2Compression supported: 0 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.merge_operator: 2024-05-21T12:16:38.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_filter: None 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55625d4adfa0) 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-05-21T12:16:38.193 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x55625d4b1090 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-05-21T12:16:38.194 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-05-21T12:16:38.195 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-05-21T12:16:38.195 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-05-21T12:16:38.195 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-05-21T12:16:38.195 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-05-21T12:16:38.195 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-05-21T12:16:38.195 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-05-21T12:16:38.196 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression: NoCompression 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.num_levels: 7 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T12:16:38.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T12:16:38.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T12:16:38.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T12:16:38.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T12:16:38.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.inplace_update_support: 0 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.bloom_locality: 0 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.max_successive_merges: 0 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T12:16:38.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T12:16:38.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.ttl: 2592000 2024-05-21T12:16:38.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T12:16:38.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T12:16:38.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T12:16:38.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.enable_blob_files: false 2024-05-21T12:16:38.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.min_blob_size: 0 2024-05-21T12:16:38.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T12:16:38.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e466bde0-e364-4e97-9197-821aa6a18c3e 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716293797893278, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716293797894168, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85883, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 261, "table_properties": {"data_size": 84030, "index_size": 237, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10250, "raw_average_key_size": 46, "raw_value_size": 78175, "raw_average_value_size": 355, "num_data_blocks": 11, "num_entries": 220, "num_filter_entries": 220, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716293797, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e466bde0-e364-4e97-9197-821aa6a18c3e", "db_session_id": "7WCNIY7NL15O9VY4Z919", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-21T12:16:38.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716293797894258, "job": 1, "event": "recovery_finished"} 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55625d5a6000 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: DB pointer 0x55625d592000 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T12:16:38.205 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 85.73 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 85.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T12:16:38.206 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 154.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 26.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 26.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T12:16:38.207 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x55625d4b1090#2 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a@-1(???) e1 preinit fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a@-1(???).mds e1 new map 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a@-1(???).mds e1 print_map 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-05-21T12:16:38.208 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: monmap epoch 1 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: last_changed 2024-05-21T12:16:29.401381+0000 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: created 2024-05-21T12:16:29.401381+0000 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: min_mon_release 19 (squid) 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: election_strategy: 1 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: fsmap 2024-05-21T12:16:38.209 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: osdmap e1: 0 total, 0 up, 0 in 2024-05-21T12:16:38.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 ceph-mon[30165]: mgrmap e1: no daemons active 2024-05-21T12:16:38.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 bash[30144]: 80d88adb7cd3fb8ffc9a79f419c020917186deff3a8ad2862cff51735a3090b3 2024-05-21T12:16:38.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:37 smithi045 systemd[1]: Started Ceph mon.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:16:39.036 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-21T12:16:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:16:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-05-21T12:16:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-21T12:16:39.058 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-21T12:16:39.327 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T12:16:39.327 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-21T12:16:39.479 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968.target.wants/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service. 2024-05-21T12:16:39.815 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:39 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/199032448' entity='client.admin' 2024-05-21T12:16:39.985 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T12:16:39.985 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-05-21T12:16:39.993 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T12:16:39.993 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-05-21T12:16:39.993 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-05-21T12:16:39.993 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T12:16:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T12:16:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-05-21T12:16:40.010 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T12:16:40.010 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-05-21T12:16:40.010 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-05-21T12:16:40.010 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-21T12:16:40.010 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-05-21T12:16:40.010 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-05-21T12:16:40.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:39 smithi045 systemd[1]: Started Ceph mgr.x for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:16:40.365 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:40.069+0000 7fb04b84d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T12:16:40.365 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:40.268+0000 7fb04b84d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T12:16:40.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:40 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/920224922' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-05-21T12:16:40.708 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:40.376+0000 7fb04b84d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T12:16:41.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:40.707+0000 7fb04b84d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T12:16:41.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:40.797+0000 7fb04b84d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T12:16:41.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:41.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-05-21T12:16:41.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "c7e3752a-176b-11ef-bc9a-c7b262605968", 2024-05-21T12:16:41.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T12:16:41.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T12:16:41.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T12:16:41.122 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T12:16:41.123 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T12:16:32.935889+0000", 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T12:16:41.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:41.125 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T12:16:41.125 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-05-21T12:16:41.125 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-05-21T12:16:42.430 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:42.076+0000 7fb04b84d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T12:16:42.430 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:42.252+0000 7fb04b84d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T12:16:42.430 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:42.428+0000 7fb04b84d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T12:16:43.038 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:42.727+0000 7fb04b84d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T12:16:43.038 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:42.823+0000 7fb04b84d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T12:16:43.038 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:43.036+0000 7fb04b84d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T12:16:43.305 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:43.240+0000 7fb04b84d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T12:16:43.892 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:43 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/4187193789' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-05-21T12:16:44.175 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:43.890+0000 7fb04b84d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T12:16:44.176 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:44.122+0000 7fb04b84d200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "c7e3752a-176b-11ef-bc9a-c7b262605968", 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T12:16:44.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:44.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T12:16:44.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T12:16:32.935889+0000", 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:44.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T12:16:44.189 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-05-21T12:16:44.189 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-05-21T12:16:45.052 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:44.565+0000 7fb04b84d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T12:16:45.052 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:44.666+0000 7fb04b84d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T12:16:45.052 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:45.050+0000 7fb04b84d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T12:16:45.778 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:45.423+0000 7fb04b84d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T12:16:45.778 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:45.520+0000 7fb04b84d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T12:16:46.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:45.777+0000 7fb04b84d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T12:16:46.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:45.916+0000 7fb04b84d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T12:16:46.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:46.016+0000 7fb04b84d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T12:16:46.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:46.361+0000 7fb04b84d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T12:16:46.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:46.460+0000 7fb04b84d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T12:16:47.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:46 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/4089270386' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-05-21T12:16:47.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:47.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-05-21T12:16:47.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "c7e3752a-176b-11ef-bc9a-c7b262605968", 2024-05-21T12:16:47.290 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:47.291 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T12:16:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T12:16:47.293 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T12:16:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T12:16:32.935889+0000", 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T12:16:47.295 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-05-21T12:16:47.296 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (3/15)... 2024-05-21T12:16:47.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:47.102+0000 7fb04b84d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T12:16:47.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:47.330+0000 7fb04b84d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T12:16:48.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: Activating manager daemon x 2024-05-21T12:16:48.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: mgrmap e2: x(active, starting, since 0.0033632s) 2024-05-21T12:16:48.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-05-21T12:16:48.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-05-21T12:16:48.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-05-21T12:16:48.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:16:48.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-05-21T12:16:48.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: Manager daemon x is now available 2024-05-21T12:16:48.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-05-21T12:16:48.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-05-21T12:16:48.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' 2024-05-21T12:16:48.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' 2024-05-21T12:16:48.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:47 smithi045 ceph-mon[30165]: from='mgr.14100 172.21.15.45:0/2676160760' entity='mgr.x' 2024-05-21T12:16:49.610 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:49 smithi045 ceph-mon[30165]: mgrmap e3: x(active, since 1.00757s) 2024-05-21T12:16:50.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:50 smithi045 ceph-mon[30165]: mgrmap e4: x(active, since 2s) 2024-05-21T12:16:50.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:50 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2092156830' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-05-21T12:16:50.599 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:50.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-05-21T12:16:50.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "c7e3752a-176b-11ef-bc9a-c7b262605968", 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-05-21T12:16:50.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T12:16:50.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T12:16:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T12:16:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T12:16:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T12:16:50.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:50.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T12:16:50.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T12:16:50.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T12:16:50.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T12:16:50.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T12:16:50.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T12:16:50.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T12:16:50.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T12:16:32.935889+0000", 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-05-21T12:16:50.610 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-05-21T12:16:51.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:51 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1550703595' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-05-21T12:16:51.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:51.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T12:16:51.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:16:51.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-21T12:16:51.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T12:16:51.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T12:16:51.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T12:16:51.775 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-05-21T12:16:52.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:52 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/918182731' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-05-21T12:16:52.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: ignoring --setuser ceph since I am not root 2024-05-21T12:16:52.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: ignoring --setgroup ceph since I am not root 2024-05-21T12:16:52.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:52.685+0000 7fdf32eed200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T12:16:53.193 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:52.879+0000 7fdf32eed200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T12:16:53.193 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:52.987+0000 7fdf32eed200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T12:16:53.454 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:53.320+0000 7fdf32eed200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T12:16:53.455 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:53.412+0000 7fdf32eed200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T12:16:53.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:53 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/918182731' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-21T12:16:53.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:53 smithi045 ceph-mon[30165]: mgrmap e5: x(active, since 5s) 2024-05-21T12:16:54.344 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-05-21T12:16:54.345 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-21T12:16:54.345 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T12:16:54.345 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T12:16:54.345 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T12:16:54.345 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-05-21T12:16:54.345 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-05-21T12:16:54.345 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-05-21T12:16:54.686 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:54 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/380517910' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-05-21T12:16:55.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:54 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:54.693+0000 7fdf32eed200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T12:16:55.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:54 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:54.875+0000 7fdf32eed200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T12:16:55.054 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:55.052+0000 7fdf32eed200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T12:16:55.664 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:55.355+0000 7fdf32eed200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T12:16:55.664 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:55.451+0000 7fdf32eed200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T12:16:55.664 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:55.662+0000 7fdf32eed200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T12:16:56.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:55.864+0000 7fdf32eed200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T12:16:56.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:56.496+0000 7fdf32eed200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T12:16:56.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:56.726+0000 7fdf32eed200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T12:16:57.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:57.170+0000 7fdf32eed200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T12:16:57.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:57.271+0000 7fdf32eed200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T12:16:58.034 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:57.657+0000 7fdf32eed200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T12:16:58.034 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:58 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:58.032+0000 7fdf32eed200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T12:16:58.317 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:58 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:58.129+0000 7fdf32eed200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T12:16:58.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:58 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:58.385+0000 7fdf32eed200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T12:16:58.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:58 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:58.526+0000 7fdf32eed200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T12:16:58.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:58 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:58.626+0000 7fdf32eed200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T12:16:59.316 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:58 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:58.971+0000 7fdf32eed200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T12:16:59.317 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:59.068+0000 7fdf32eed200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T12:16:59.983 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:59.701+0000 7fdf32eed200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T12:16:59.983 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:16:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:16:59.928+0000 7fdf32eed200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: Active manager daemon x restarted 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: Activating manager daemon x 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: osdmap e2: 0 total, 0 up, 0 in 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: mgrmap e6: x(active, starting, since 0.00330959s) 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-05-21T12:17:00.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-05-21T12:17:00.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: Manager daemon x is now available 2024-05-21T12:17:00.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:00.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:00.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:00.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:16:59 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:01.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:00 smithi045 ceph-mon[30165]: Found migration_current of "None". Setting to last migration. 2024-05-21T12:17:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:00 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-05-21T12:17:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:00 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-05-21T12:17:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:00 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:00 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:00 smithi045 ceph-mon[30165]: mgrmap e7: x(active, since 1.00684s) 2024-05-21T12:17:01.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-05-21T12:17:01.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-21T12:17:01.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T12:17:01.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-05-21T12:17:01.543 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-05-21T12:17:01.543 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-05-21T12:17:02.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:01 smithi045 ceph-mon[30165]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T12:17:02.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:01 smithi045 ceph-mon[30165]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: [21/May/2024:12:17:01] ENGINE Bus STARTING 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: mgrmap e8: x(active, since 2s) 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: [21/May/2024:12:17:02] ENGINE Serving on http://172.21.15.45:8765 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: [21/May/2024:12:17:02] ENGINE Serving on https://172.21.15.45:7150 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: [21/May/2024:12:17:02] ENGINE Bus STARTED 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:03.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:02 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:04.027 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:03 smithi045 ceph-mon[30165]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:04.030 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-21T12:17:04.030 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-05-21T12:17:05.202 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: Generating public/private rsa key pair. 2024-05-21T12:17:05.202 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: Your identification has been saved in /tmp/tmp_e6rasc7/key. 2024-05-21T12:17:05.202 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: Your public key has been saved in /tmp/tmp_e6rasc7/key.pub. 2024-05-21T12:17:05.202 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: The key fingerprint is: 2024-05-21T12:17:05.202 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: SHA256:jKW06kSyGiO17aLTUg11aT1yOpNxXkVxctHiAlYsnWs ceph-c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:17:05.202 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: The key's randomart image is: 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: +---[RSA 3072]----+ 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: | o **o+o | 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: | . * = = ++. .| 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: | . o.O.+ o o . | 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: | . .=*. E . | 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: | oo. +oS . . | 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: | ..*.. | 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: |+oo + | 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: |++o+ | 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: |o+ .o | 2024-05-21T12:17:05.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: +----[SHA256]-----+ 2024-05-21T12:17:06.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:05 smithi045 ceph-mon[30165]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:06.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:05 smithi045 ceph-mon[30165]: Generating ssh key... 2024-05-21T12:17:06.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:05 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:06.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:05 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:06.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:17:06.668 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-21T12:17:06.669 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-05-21T12:17:06.669 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-05-21T12:17:06.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:06 smithi045 ceph-mon[30165]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:07.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:07 smithi045 ceph-mon[30165]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:09.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:08 smithi045 ceph-mon[30165]: Deploying cephadm binary to smithi045 2024-05-21T12:17:12.218 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-05-21T12:17:12.218 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-05-21T12:17:12.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:12 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:12.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:12 smithi045 ceph-mon[30165]: Added host smithi045 2024-05-21T12:17:12.831 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:12 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:13.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-21T12:17:13.847 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-05-21T12:17:14.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:13 smithi045 ceph-mon[30165]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:14.140 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:13 smithi045 ceph-mon[30165]: Saving service mon spec with placement count:5 2024-05-21T12:17:14.140 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:13 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:15.415 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-21T12:17:15.666 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:15 smithi045 ceph-mon[30165]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:15.666 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:15 smithi045 ceph-mon[30165]: Saving service mgr spec with placement count:2 2024-05-21T12:17:15.666 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:15 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:15.666 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:15 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:15.666 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:15 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:17.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:17 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/4077240578' entity='client.admin' 2024-05-21T12:17:17.284 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:17 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:18.198 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-05-21T12:17:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:18 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/828569539' entity='client.admin' 2024-05-21T12:17:18.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:18 smithi045 ceph-mon[30165]: from='mgr.14120 172.21.15.45:0/4116631891' entity='mgr.x' 2024-05-21T12:17:20.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:19 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3534117943' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-05-21T12:17:20.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: ignoring --setuser ceph since I am not root 2024-05-21T12:17:20.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: ignoring --setgroup ceph since I am not root 2024-05-21T12:17:20.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:19.888+0000 7f83fe50c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T12:17:20.334 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:20.081+0000 7f83fe50c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T12:17:20.334 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:20.188+0000 7f83fe50c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T12:17:20.610 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:20.515+0000 7f83fe50c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T12:17:20.610 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:20.608+0000 7f83fe50c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T12:17:21.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:20 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3534117943' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-21T12:17:21.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:20 smithi045 ceph-mon[30165]: mgrmap e9: x(active, since 19s) 2024-05-21T12:17:21.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-05-21T12:17:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-21T12:17:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T12:17:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T12:17:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T12:17:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-05-21T12:17:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-05-21T12:17:21.653 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-05-21T12:17:21.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:21 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2790662640' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-05-21T12:17:22.276 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:21 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:21.905+0000 7f83fe50c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T12:17:22.276 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:22.093+0000 7f83fe50c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T12:17:22.276 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:22.273+0000 7f83fe50c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T12:17:22.889 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:22.579+0000 7f83fe50c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T12:17:22.889 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:22.676+0000 7f83fe50c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T12:17:22.889 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:22.888+0000 7f83fe50c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T12:17:23.316 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:23.091+0000 7f83fe50c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T12:17:24.066 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:23.728+0000 7f83fe50c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T12:17:24.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:23.958+0000 7f83fe50c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T12:17:24.816 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:24.405+0000 7f83fe50c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T12:17:24.817 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:24.506+0000 7f83fe50c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T12:17:25.271 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:24.894+0000 7f83fe50c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T12:17:25.271 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:25.269+0000 7f83fe50c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T12:17:25.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:25.368+0000 7f83fe50c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T12:17:26.066 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:25.625+0000 7f83fe50c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T12:17:26.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:25.767+0000 7f83fe50c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T12:17:26.067 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:25.868+0000 7f83fe50c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T12:17:26.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:26.211+0000 7f83fe50c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T12:17:26.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:26.310+0000 7f83fe50c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T12:17:27.231 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:26.949+0000 7f83fe50c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T12:17:27.232 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:17:27 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:17:27.178+0000 7f83fe50c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: Active manager daemon x restarted 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: Activating manager daemon x 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: osdmap e3: 0 total, 0 up, 0 in 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: mgrmap e10: x(active, starting, since 0.00326371s) 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: Manager daemon x is now available 2024-05-21T12:17:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:27 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:28.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-05-21T12:17:28.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-05-21T12:17:28.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:28 smithi045 ceph-mon[30165]: mgrmap e11: x(active, since 1.00626s) 2024-05-21T12:17:29.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-05-21T12:17:29.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-21T12:17:29.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T12:17:29.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-05-21T12:17:29.155 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-05-21T12:17:29.155 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2024-05-21T12:17:29.505 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:29 smithi045 ceph-mon[30165]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T12:17:29.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:29 smithi045 ceph-mon[30165]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T12:17:29.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:29 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:30.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:30 smithi045 ceph-mon[30165]: [21/May/2024:12:17:29] ENGINE Bus STARTING 2024-05-21T12:17:30.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:30 smithi045 ceph-mon[30165]: [21/May/2024:12:17:29] ENGINE Serving on http://172.21.15.45:8765 2024-05-21T12:17:30.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:30 smithi045 ceph-mon[30165]: [21/May/2024:12:17:29] ENGINE Serving on https://172.21.15.45:7150 2024-05-21T12:17:30.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:30 smithi045 ceph-mon[30165]: [21/May/2024:12:17:29] ENGINE Bus STARTED 2024-05-21T12:17:30.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:30 smithi045 ceph-mon[30165]: mgrmap e12: x(active, since 2s) 2024-05-21T12:17:30.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:30.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:30.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:30.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-21T12:17:30.521 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-05-21T12:17:31.356 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:31 smithi045 ceph-mon[30165]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7FJshObJqwgUaWV13ZLrRur0UW1rGeIp.XLQoPnxzEFJ2V4xwAmaG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716293851, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-21T12:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-05-21T12:17:32.732 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:32 smithi045 ceph-mon[30165]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:32.732 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-05-21T12:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T12:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-05-21T12:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T12:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-05-21T12:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-05-21T12:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-21T12:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-05-21T12:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2024-05-21T12:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-05-21T12:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: Password: 616iol5ygw 2024-05-21T12:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config directory 2024-05-21T12:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2024-05-21T12:17:33.703 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:33 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1701957027' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-05-21T12:17:34.532 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:34 smithi045 ceph-mon[30165]: mgrmap e13: x(active, since 6s) 2024-05-21T12:17:36.050 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-21T12:17:36.050 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-21T12:17:36.050 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:36.050 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:17:36.050 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:36.050 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:36.051 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-05-21T12:17:36.092 INFO:tasks.cephadm:Fetching config... 2024-05-21T12:17:36.092 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:17:36.092 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-21T12:17:36.109 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-21T12:17:36.109 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:17:36.109 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-21T12:17:36.165 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-21T12:17:36.166 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:17:36.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-21T12:17:36.232 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-21T12:17:36.232 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:17:36.233 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-21T12:17:36.289 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-21T12:17:36.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T12:17:36.367 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:17:36.379 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T12:17:36.419 INFO:teuthology.orchestra.run.smithi072.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:17:36.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T12:17:36.474 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:17:36.487 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T12:17:36.528 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv6GAE7wNT7UUgoIX2cP0qTha5z1Y/QnFIj4PsByJUVQIGewhdBmc301L1u0W4oNU8IXSRoPSPVPrtSBupc51huLXaL4nxITyghhXsi3jWKImnD06qHUQ73G5XGVyWnvgrSuvgHxAP46Wojnzf3i0pjF9yZ3bwZYHD0MlqgVEnNXpaId1ONiZh/9B2pyO/N4h5ZlL8xLSxYQH4Fva9yUsg8Tqis431lNe08WxULvOGX/MfghkugbWKnZpAMrm4UF/Hz/ZQYuafmSDZcMOyUH/Z3/WdvCl+XMN3tIb5WTxSioGiCjjeAwtUjslfKWlPJ4SA3r8Dn8t9un1pGS4R8oPm3Eyia4PnAXXMKG9LG1vmuO5DQmdvFlmui6H9pti2Cf76xIRIk7WoUsG3gROYwKjgxW7rfx0Ke8qY+90+rM4qVOee6KZBpswE65got8KtGICkyCuwW7w8KljFX3Qh0URqV9go2l0RDwY2eCgiPNrnmKJg6/7n3pNHfCfruqFTK6s= ceph-c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:17:36.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-21T12:17:36.787 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:17:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:36 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/479081658' entity='client.admin' 2024-05-21T12:17:39.641 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-21T12:17:39.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-21T12:17:39.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:39 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2940426300' entity='client.admin' 2024-05-21T12:17:40.233 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:17:42.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:42.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:42.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"} : dispatch 2024-05-21T12:17:42.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:42.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:42.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:43.314 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi072 2024-05-21T12:17:43.314 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:17:43.314 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/etc/ceph/ceph.conf 2024-05-21T12:17:43.334 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:17:43.334 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:17:43.392 INFO:tasks.cephadm:Adding host smithi072 to orchestrator... 2024-05-21T12:17:43.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch host add smithi072 2024-05-21T12:17:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:17:43.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:43 smithi045 ceph-mon[30165]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:43.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:43.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:43.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:17:43.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:17:43.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:43 smithi045 ceph-mon[30165]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-21T12:17:44.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:44 smithi045 ceph-mon[30165]: Updating smithi045:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:17:44.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:44 smithi045 ceph-mon[30165]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:17:44.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:44 smithi045 ceph-mon[30165]: Updating smithi045:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T12:17:44.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:44.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:44.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:46.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:45 smithi045 ceph-mon[30165]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi072", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:47.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:46 smithi045 ceph-mon[30165]: Deploying cephadm binary to smithi072 2024-05-21T12:17:48.460 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:48 smithi045 ceph-mon[30165]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:17:49.506 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi072' with addr '172.21.15.72' 2024-05-21T12:17:50.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T12:17:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:17:50.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:50 smithi045 ceph-mon[30165]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:17:50.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:50.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:50 smithi045 ceph-mon[30165]: Added host smithi072 2024-05-21T12:17:50.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:50.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:51.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:51 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:51.977 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:17:51.978 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}] 2024-05-21T12:17:52.499 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi092 2024-05-21T12:17:52.499 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:17:52.500 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.conf 2024-05-21T12:17:52.518 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:17:52.518 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:17:52.574 INFO:tasks.cephadm:Adding host smithi092 to orchestrator... 2024-05-21T12:17:52.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch host add smithi092 2024-05-21T12:17:52.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:52 smithi045 ceph-mon[30165]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:17:52.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:52 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:52.826 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:17:53.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:53 smithi045 ceph-mon[30165]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:17:54.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:54 smithi045 ceph-mon[30165]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:17:55.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:55 smithi045 ceph-mon[30165]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:17:55.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:55.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:55.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:55.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:55.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi072", "name": "osd_memory_target"} : dispatch 2024-05-21T12:17:55.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:17:55.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:17:56.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:56 smithi045 ceph-mon[30165]: Deploying cephadm binary to smithi092 2024-05-21T12:17:56.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:56 smithi045 ceph-mon[30165]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-21T12:17:56.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:56 smithi045 ceph-mon[30165]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:17:56.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:56 smithi045 ceph-mon[30165]: Updating smithi072:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:17:56.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:56 smithi045 ceph-mon[30165]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:17:58.777 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi092' with addr '172.21.15.92' 2024-05-21T12:17:58.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:58 smithi045 ceph-mon[30165]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:17:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:59.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T12:17:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:17:59.767 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:59 smithi045 ceph-mon[30165]: Updating smithi072:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T12:17:59.767 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:17:59.767 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:17:59.767 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:17:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:00.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:00 smithi045 ceph-mon[30165]: Added host smithi092 2024-05-21T12:18:00.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:00 smithi045 ceph-mon[30165]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:00.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:00 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:01.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:18:01.332 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}] 2024-05-21T12:18:01.966 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-05-21T12:18:01.966 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:18:01.967 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-05-21T12:18:01.987 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:18:01.988 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:02.046 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-05-21T12:18:02.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch host add smithi137 2024-05-21T12:18:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:18:02.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:02 smithi045 ceph-mon[30165]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:02.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:02 smithi045 ceph-mon[30165]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:18:02.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:04 smithi045 ceph-mon[30165]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:05.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:05.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:05.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:05.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:05.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:05.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-05-21T12:18:05.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:05.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:05.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-21T12:18:05.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: Deploying cephadm binary to smithi137 2024-05-21T12:18:05.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:05 smithi045 ceph-mon[30165]: Updating smithi092:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:06.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:06 smithi045 ceph-mon[30165]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:06.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:06 smithi045 ceph-mon[30165]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:08.189 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-05-21T12:18:08.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:08 smithi045 ceph-mon[30165]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:08.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:08 smithi045 ceph-mon[30165]: Updating smithi092:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T12:18:08.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:08.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:08.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:08.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:08.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:08.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T12:18:09.071 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:18:09.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:09 smithi045 ceph-mon[30165]: Added host smithi137 2024-05-21T12:18:09.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:09 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:09.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:09 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:10.807 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:18:10.808 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-05-21T12:18:10.808 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:10 smithi045 ceph-mon[30165]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:11.391 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-21T12:18:11.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-21T12:18:11.643 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:18:12.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:12.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:11 smithi045 ceph-mon[30165]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:18:12.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:11 smithi045 ceph-mon[30165]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:14.248 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-05-21T12:18:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2015252697' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-05-21T12:18:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-05-21T12:18:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:14.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:14 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:14.928 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-05-21T12:18:14.928 INFO:tasks.cephadm:Adding mon.b on smithi072 2024-05-21T12:18:14.928 INFO:tasks.cephadm:Adding mon.c on smithi092 2024-05-21T12:18:14.928 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c' 2024-05-21T12:18:15.156 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:15.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-21T12:18:15.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: Updating smithi137:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:15.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2015252697' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T12:18:15.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T12:18:15.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:15.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: Updating smithi137:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T12:18:15.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:15.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:15.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:15 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:16.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:16 smithi045 ceph-mon[30165]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:16.731 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mon update... 2024-05-21T12:18:17.196 DEBUG:teuthology.orchestra.run.smithi072:mon.b> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.b.service 2024-05-21T12:18:17.199 DEBUG:teuthology.orchestra.run.smithi092:mon.c> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.c.service 2024-05-21T12:18:17.202 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T12:18:17.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T12:18:17.948 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:18.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:18.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:18:18.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:18.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:18.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:18.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:18.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:18.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:18.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:17 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:19.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:18 smithi045 ceph-mon[30165]: Deploying daemon mon.c on smithi092 2024-05-21T12:18:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:18 smithi045 ceph-mon[30165]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:20.376 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:20 smithi092 systemd[1]: Starting Ceph mon.c for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:18:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:18:21.061 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":1,"fsid":"c7e3752a-176b-11ef-bc9a-c7b262605968","modified":"2024-05-21T12:16:29.401381Z","created":"2024-05-21T12:16:29.401381Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-21T12:18:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 1 2024-05-21T12:18:21.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:20 smithi045 ceph-mon[30165]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:20 smithi092 podman[33634]: 2024-05-21 12:18:20.823353337 +0000 UTC m=+0.017451954 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:20 smithi092 podman[33634]: 2024-05-21 12:18:20.92738675 +0000 UTC m=+0.121485351 container create 9de20c30070169f0e6746e116b2a2bde09001ea26f6dd900bc3bf8fd84305376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=ceph-nvmeof-mon-b0c764b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 podman[33634]: 2024-05-21 12:18:21.10523965 +0000 UTC m=+0.299338268 container init 9de20c30070169f0e6746e116b2a2bde09001ea26f6dd900bc3bf8fd84305376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux ) 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 podman[33634]: 2024-05-21 12:18:21.108781303 +0000 UTC m=+0.302879913 container start 9de20c30070169f0e6746e116b2a2bde09001ea26f6dd900bc3bf8fd84305376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.build-date=20240514) 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: set uid:gid to 167:167 (ceph:ceph) 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev), process ceph-mon, pid 2 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pidfile_write: ignore empty --pid-file 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: load: jerasure load: lrc 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: RocksDB version: 7.9.2 2024-05-21T12:18:21.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Git sha 0 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Compile date 2024-05-16 09:00:48 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: DB SUMMARY 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: DB Session ID: RZZQS76I7JBYGZ3MC4X1 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: CURRENT file: CURRENT 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.error_if_exists: 0 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.create_if_missing: 0 2024-05-21T12:18:21.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.paranoid_checks: 1 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.env: 0x55e5efc030c0 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.info_log: 0x55e5f06d3de0 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.statistics: (nil) 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.use_fsync: 0 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_log_file_size: 0 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T12:18:21.256 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.allow_fallocate: 1 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.use_direct_reads: 0 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.db_log_dir: 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.wal_dir: 2024-05-21T12:18:21.257 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.write_buffer_manager: 0x55e5f06a41e0 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T12:18:21.258 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T12:18:21.260 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.unordered_write: 0 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.row_cache: None 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.wal_filter: None 2024-05-21T12:18:21.261 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.two_write_queues: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.wal_compression: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.atomic_flush: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.log_readahead_size: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T12:18:21.262 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_background_jobs: 2 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_background_compactions: -1 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_subcompactions: 1 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T12:18:21.263 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_open_files: -1 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_background_flushes: -1 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Compression algorithms supported: 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: kZSTD supported: 0 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: kXpressCompression supported: 0 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: kZlibCompression supported: 1 2024-05-21T12:18:21.264 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: kSnappyCompression supported: 1 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: kLZ4Compression supported: 1 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: kBZip2Compression supported: 0 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.merge_operator: 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_filter: None 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T12:18:21.265 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e5f06d3fa0) 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: index_type: 0 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: data_block_index_type: 0 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: index_shortening: 1 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: checksum: 4 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: no_block_cache: 0 2024-05-21T12:18:21.266 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_cache: 0x55e5f06d7090 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_cache_options: 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: capacity : 536870912 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: num_shard_bits : 4 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: strict_capacity_limit : 0 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_cache_compressed: (nil) 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: persistent_cache: (nil) 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_size: 4096 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_size_deviation: 10 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_restart_interval: 16 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: index_block_restart_interval: 1 2024-05-21T12:18:21.267 INFO:journalctl@ceph.mon.c.smithi092.stdout: metadata_block_size: 4096 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: partition_filters: 0 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: use_delta_encoding: 1 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: filter_policy: bloomfilter 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: whole_key_filtering: 1 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: verify_compression: 0 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: format_version: 5 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: enable_index_compression: 1 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: block_align: 0 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: max_auto_readahead_size: 262144 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: prepopulate_block_cache: 0 2024-05-21T12:18:21.268 INFO:journalctl@ceph.mon.c.smithi092.stdout: initial_auto_readahead_size: 8192 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression: NoCompression 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.num_levels: 7 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T12:18:21.269 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T12:18:21.270 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T12:18:21.271 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T12:18:21.272 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.inplace_update_support: 0 2024-05-21T12:18:21.273 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.bloom_locality: 0 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.max_successive_merges: 0 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.ttl: 2592000 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T12:18:21.274 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.enable_blob_files: false 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.min_blob_size: 0 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-21T12:18:21.275 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: db59b3ea-974e-4e6c-88b5-ce6096b991d7 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716293901138980, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716293901139402, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716293901, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "db59b3ea-974e-4e6c-88b5-ce6096b991d7", "db_session_id": "RZZQS76I7JBYGZ3MC4X1", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716293901139480, "job": 1, "event": "recovery_finished"} 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e5f07cc000 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: DB pointer 0x55e5f07b8000 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** DB Stats ** 2024-05-21T12:18:21.276 INFO:journalctl@ceph.mon.c.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-21T12:18:21.277 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** Compaction Stats [default] ** 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T12:18:21.278 INFO:journalctl@ceph.mon.c.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T12:18:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T12:18:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T12:18:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T12:18:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout: Block cache BinnedLRUCache@0x55e5f06d7090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-21T12:18:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-21T12:18:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-05-21T12:18:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: using public_addr v2:172.21.15.92:0/0 -> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: starting mon.c rank -1 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(???) e0 preinit fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).mds e1 new map 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout: e1 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout: legacy client fscid: -1 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout: 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout: No filesystems configured 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-21T12:18:21.280 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:18:21.281 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi072", "name": "osd_memory_target"} : dispatch 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Deploying cephadm binary to smithi092 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-21T12:18:21.282 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi072:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi072:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Added host smithi092 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.283 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"} : dispatch 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:21.284 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Deploying cephadm binary to smithi137 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi092:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi092:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Added host smithi137 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.285 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2015252697' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-05-21T12:18:21.286 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi137:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2015252697' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Updating smithi137:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:18:21.287 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: Deploying daemon mon.c on smithi092 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:21.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 ceph-mon[33665]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-21T12:18:22.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 bash[33634]: 9de20c30070169f0e6746e116b2a2bde09001ea26f6dd900bc3bf8fd84305376 2024-05-21T12:18:22.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:21 smithi092 systemd[1]: Started Ceph mon.c for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:18:22.593 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T12:18:22.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T12:18:22.834 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:18:26.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:18:26.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: mon.a calling monitor election 2024-05-21T12:18:26.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:26.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: mon.c calling monitor election 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: monmap epoch 2 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: last_changed 2024-05-21T12:18:21.165860+0000 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: created 2024-05-21T12:16:29.401381+0000 2024-05-21T12:18:26.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: min_mon_release 19 (squid) 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: election_strategy: 1 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: fsmap 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: mgrmap e13: x(active, since 59s) 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: overall HEALTH_OK 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:26.505 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:26 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:26.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:18:26.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: mon.a calling monitor election 2024-05-21T12:18:26.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:26.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: mon.c calling monitor election 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: monmap epoch 2 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:26.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: last_changed 2024-05-21T12:18:21.165860+0000 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: created 2024-05-21T12:16:29.401381+0000 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: min_mon_release 19 (squid) 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: election_strategy: 1 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: fsmap 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: mgrmap e13: x(active, since 59s) 2024-05-21T12:18:26.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: overall HEALTH_OK 2024-05-21T12:18:26.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:26.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:26.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:26.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:26.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:26 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:27.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:27 smithi092 ceph-mon[33665]: Deploying daemon mon.b on smithi072 2024-05-21T12:18:27.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:27 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:27.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:18:27.550 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":2,"fsid":"c7e3752a-176b-11ef-bc9a-c7b262605968","modified":"2024-05-21T12:18:21.165860Z","created":"2024-05-21T12:16:29.401381Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T12:18:27.553 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 2 2024-05-21T12:18:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:27 smithi045 ceph-mon[30165]: Deploying daemon mon.b on smithi072 2024-05-21T12:18:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:27 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:28.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:28 smithi092 ceph-mon[33665]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:28 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/2460703301' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-05-21T12:18:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:28 smithi045 ceph-mon[30165]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:28 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/2460703301' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-05-21T12:18:29.141 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T12:18:29.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T12:18:29.380 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:18:30.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:29 smithi072 systemd[1]: Started Ceph mon.b for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:18:35.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:18:35.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: mon.a calling monitor election 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: mon.c calling monitor election 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: mon.b calling monitor election 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: monmap epoch 3 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: last_changed 2024-05-21T12:18:29.960463+0000 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: created 2024-05-21T12:16:29.401381+0000 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: min_mon_release 19 (squid) 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: election_strategy: 1 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: fsmap 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T12:18:35.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: mgrmap e13: x(active, since 67s) 2024-05-21T12:18:35.320 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: overall HEALTH_OK 2024-05-21T12:18:35.320 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.320 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.320 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.320 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:18:35.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: mon.a calling monitor election 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: mon.c calling monitor election 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: mon.b calling monitor election 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: monmap epoch 3 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: last_changed 2024-05-21T12:18:29.960463+0000 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: created 2024-05-21T12:16:29.401381+0000 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: min_mon_release 19 (squid) 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: election_strategy: 1 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: fsmap 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: mgrmap e13: x(active, since 67s) 2024-05-21T12:18:35.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: overall HEALTH_OK 2024-05-21T12:18:35.419 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.419 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.419 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.419 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:18:35.488 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"fsid":"c7e3752a-176b-11ef-bc9a-c7b262605968","modified":"2024-05-21T12:18:29.960463Z","created":"2024-05-21T12:16:29.401381Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-21T12:18:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:18:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: mon.a calling monitor election 2024-05-21T12:18:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:18:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: mon.c calling monitor election 2024-05-21T12:18:35.488 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: mon.b calling monitor election 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: monmap epoch 3 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: fsid c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: last_changed 2024-05-21T12:18:29.960463+0000 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: created 2024-05-21T12:16:29.401381+0000 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: min_mon_release 19 (squid) 2024-05-21T12:18:35.489 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: election_strategy: 1 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: fsmap 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: mgrmap e13: x(active, since 67s) 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: overall HEALTH_OK 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.490 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 3 2024-05-21T12:18:36.115 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:36 smithi092 ceph-mon[33665]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:36.115 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:36.115 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:36 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/3510618047' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-05-21T12:18:36.115 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:36.140 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-21T12:18:36.141 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-21T12:18:36.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:36 smithi045 ceph-mon[30165]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:36.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:36.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:36 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/3510618047' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-05-21T12:18:36.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:18:36.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:36 smithi072 ceph-mon[33912]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:36.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:36.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:36 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/3510618047' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-05-21T12:18:36.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:18:37.732 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.732 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.732 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:37.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:37.918 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.918 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.918 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.918 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.918 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.919 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.919 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.919 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.919 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:37.919 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:38.198 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-05-21T12:18:38.198 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:18:38.199 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] 2024-05-21T12:18:39.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-21T12:18:39.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-21T12:18:39.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-21T12:18:39.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-21T12:18:39.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Updating smithi045:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Updating smithi072:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Updating smithi092:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Updating smithi137:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: Reconfiguring daemon mon.a on smithi045 2024-05-21T12:18:39.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:38 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/334270128' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Updating smithi045:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Updating smithi072:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Updating smithi092:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Updating smithi137:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: Reconfiguring daemon mon.a on smithi045 2024-05-21T12:18:39.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:38 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/334270128' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:39.108 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-21T12:18:39.108 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-21T12:18:39.108 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-21T12:18:39.108 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-21T12:18:39.108 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Updating smithi045:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.108 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Updating smithi072:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Updating smithi092:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Updating smithi137:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: Reconfiguring daemon mon.a on smithi045 2024-05-21T12:18:39.109 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:38 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/334270128' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:39.735 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-21T12:18:39.735 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:18:39.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T12:18:39.767 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:18:39.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:39.836 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:18:39.836 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T12:18:39.864 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:18:39.864 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:39.928 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:18:39.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T12:18:39.961 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:18:39.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:40.031 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:18:40.031 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T12:18:40.062 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:18:40.062 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T12:18:40.131 INFO:tasks.cephadm:Adding mgr.x on smithi045 2024-05-21T12:18:40.131 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '1;smithi045=x' 2024-05-21T12:18:40.402 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:18:40.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:40 smithi072 ceph-mon[33912]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:40.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:40.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:40.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:40 smithi072 ceph-mon[33912]: Reconfiguring mon.b (monmap changed)... 2024-05-21T12:18:40.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:40.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:40.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:40.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:40 smithi072 ceph-mon[33912]: Reconfiguring daemon mon.b on smithi072 2024-05-21T12:18:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:40 smithi045 ceph-mon[30165]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:40 smithi045 ceph-mon[30165]: Reconfiguring mon.b (monmap changed)... 2024-05-21T12:18:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:40 smithi045 ceph-mon[30165]: Reconfiguring daemon mon.b on smithi072 2024-05-21T12:18:40.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:40 smithi092 ceph-mon[33665]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:40.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:40.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:40.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:40 smithi092 ceph-mon[33665]: Reconfiguring mon.b (monmap changed)... 2024-05-21T12:18:40.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:40.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:40.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:40.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:40 smithi092 ceph-mon[33665]: Reconfiguring daemon mon.b on smithi072 2024-05-21T12:18:41.977 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mgr update... 2024-05-21T12:18:42.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:41 smithi092 ceph-mon[33665]: Reconfiguring mon.c (monmap changed)... 2024-05-21T12:18:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:41 smithi092 ceph-mon[33665]: Reconfiguring daemon mon.c on smithi092 2024-05-21T12:18:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:41 smithi092 ceph-mon[33665]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:42.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:42.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:42.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:41 smithi072 ceph-mon[33912]: Reconfiguring mon.c (monmap changed)... 2024-05-21T12:18:42.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:42.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:42.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:42.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:41 smithi072 ceph-mon[33912]: Reconfiguring daemon mon.c on smithi092 2024-05-21T12:18:42.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:41 smithi072 ceph-mon[33912]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:42.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:42.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:42.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:41 smithi045 ceph-mon[30165]: Reconfiguring mon.c (monmap changed)... 2024-05-21T12:18:42.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-05-21T12:18:42.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-05-21T12:18:42.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:42.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:41 smithi045 ceph-mon[30165]: Reconfiguring daemon mon.c on smithi092 2024-05-21T12:18:42.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:41 smithi045 ceph-mon[30165]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:42.575 INFO:tasks.cephadm:Deploying OSDs... 2024-05-21T12:18:42.575 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:18:42.575 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T12:18:42.593 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T12:18:42.593 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-05-21T12:18:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T12:18:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:42.651 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-05-21T12:18:42.651 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:42.651 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:42.651 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-21 12:17:40.637027932 +0000 2024-05-21T12:18:42.651 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-21 12:15:07.299381396 +0000 2024-05-21T12:18:42.651 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-21 12:15:07.299381396 +0000 2024-05-21T12:18:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-21 12:15:07.299381396 +0000 2024-05-21T12:18:42.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T12:18:42.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-21T12:18:42.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-21T12:18:42.717 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.4845e-05 s, 5.4 MB/s 2024-05-21T12:18:42.719 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T12:18:42.776 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-05-21T12:18:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T12:18:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:42.832 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 960 Links: 1 2024-05-21T12:18:42.832 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:42.832 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:42.833 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-21 12:17:40.673027379 +0000 2024-05-21T12:18:42.833 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-21 12:15:07.298381412 +0000 2024-05-21T12:18:42.833 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-21 12:15:07.298381412 +0000 2024-05-21T12:18:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-21 12:15:07.298381412 +0000 2024-05-21T12:18:42.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T12:18:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-21T12:18:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-21T12:18:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000110242 s, 4.6 MB/s 2024-05-21T12:18:42.897 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T12:18:42.954 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-05-21T12:18:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T12:18:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:43.011 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 962 Links: 1 2024-05-21T12:18:43.011 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:43.011 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:43.011 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-21 12:17:40.707026856 +0000 2024-05-21T12:18:43.012 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-21 12:15:07.298381412 +0000 2024-05-21T12:18:43.012 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-21 12:15:07.298381412 +0000 2024-05-21T12:18:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-21 12:15:07.298381412 +0000 2024-05-21T12:18:43.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T12:18:43.075 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-21T12:18:43.075 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-21T12:18:43.075 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000112105 s, 4.6 MB/s 2024-05-21T12:18:43.076 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T12:18:43.134 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-05-21T12:18:43.191 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T12:18:43.191 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:43.192 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-05-21T12:18:43.192 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:43.192 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:43.192 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-21 12:17:40.742026318 +0000 2024-05-21T12:18:43.192 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-21 12:15:07.299381396 +0000 2024-05-21T12:18:43.192 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-21 12:15:07.299381396 +0000 2024-05-21T12:18:43.192 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-21 12:15:07.299381396 +0000 2024-05-21T12:18:43.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T12:18:43.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "1;smithi045=x", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:43.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: Saving service mgr spec with placement smithi045=x;count:1 2024-05-21T12:18:43.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:43.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:42 smithi092 ceph-mon[33665]: Reconfiguring daemon mgr.x on smithi045 2024-05-21T12:18:43.255 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "1;smithi045=x", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: Saving service mgr spec with placement smithi045=x;count:1 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-05-21T12:18:43.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:43.257 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:42 smithi045 ceph-mon[30165]: Reconfiguring daemon mgr.x on smithi045 2024-05-21T12:18:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-21T12:18:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-21T12:18:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000103046 s, 5.0 MB/s 2024-05-21T12:18:43.258 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T12:18:43.315 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:18:43.315 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T12:18:43.334 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T12:18:43.334 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2024-05-21T12:18:43.394 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T12:18:43.394 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:43.394 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 962 Links: 1 2024-05-21T12:18:43.394 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:43.394 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:43.395 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-21 12:18:29.417836142 +0000 2024-05-21T12:18:43.395 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.395 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.395 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.395 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T12:18:43.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "1;smithi045=x", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:43.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: Saving service mgr spec with placement smithi045=x;count:1 2024-05-21T12:18:43.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:18:43.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:43.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:18:43.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:43.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T12:18:43.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-05-21T12:18:43.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-05-21T12:18:43.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:43.419 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:42 smithi072 ceph-mon[33912]: Reconfiguring daemon mgr.x on smithi045 2024-05-21T12:18:43.446 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-21T12:18:43.447 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-21T12:18:43.447 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000287594 s, 1.8 MB/s 2024-05-21T12:18:43.448 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T12:18:43.507 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2024-05-21T12:18:43.566 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T12:18:43.566 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:43.566 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-05-21T12:18:43.566 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:43.566 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:43.566 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-21 12:18:29.417836142 +0000 2024-05-21T12:18:43.566 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.566 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.567 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.567 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T12:18:43.633 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-21T12:18:43.633 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-21T12:18:43.633 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000265619 s, 1.9 MB/s 2024-05-21T12:18:43.635 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T12:18:43.693 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2024-05-21T12:18:43.749 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T12:18:43.750 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:43.750 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-05-21T12:18:43.750 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:43.750 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:43.750 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-21 12:18:29.417836142 +0000 2024-05-21T12:18:43.750 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-21 12:15:11.656807021 +0000 2024-05-21T12:18:43.750 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-21 12:15:11.656807021 +0000 2024-05-21T12:18:43.750 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-21 12:15:11.656807021 +0000 2024-05-21T12:18:43.750 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T12:18:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-21T12:18:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-21T12:18:43.815 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000192836 s, 2.7 MB/s 2024-05-21T12:18:43.816 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T12:18:43.874 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2024-05-21T12:18:43.930 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T12:18:43.930 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:43.931 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-05-21T12:18:43.931 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:43.931 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:43.931 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-21 12:18:29.418836122 +0000 2024-05-21T12:18:43.931 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.931 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.931 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-21 12:15:11.655807041 +0000 2024-05-21T12:18:43.932 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T12:18:43.995 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-21T12:18:43.995 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-21T12:18:43.995 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000225088 s, 2.3 MB/s 2024-05-21T12:18:43.996 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T12:18:44.054 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:18:44.055 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T12:18:44.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:43 smithi045 ceph-mon[30165]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:44.074 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T12:18:44.074 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-21 12:18:20.240448985 +0000 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.133 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T12:18:44.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:43 smithi072 ceph-mon[33912]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:44.202 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-21T12:18:44.202 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-21T12:18:44.202 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000212836 s, 2.4 MB/s 2024-05-21T12:18:44.203 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T12:18:44.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:43 smithi092 ceph-mon[33665]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:44.263 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-05-21T12:18:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T12:18:44.322 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:44.323 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-05-21T12:18:44.323 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:44.323 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:44.323 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-21 12:18:20.240448985 +0000 2024-05-21T12:18:44.323 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.323 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.323 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T12:18:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-21T12:18:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-21T12:18:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000321328 s, 1.6 MB/s 2024-05-21T12:18:44.391 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T12:18:44.451 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-05-21T12:18:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T12:18:44.509 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:44.509 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-05-21T12:18:44.509 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:44.509 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:44.509 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-21 12:18:20.240448985 +0000 2024-05-21T12:18:44.509 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.509 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.510 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T12:18:44.576 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-21T12:18:44.577 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-21T12:18:44.577 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000237683 s, 2.2 MB/s 2024-05-21T12:18:44.578 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T12:18:44.637 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-05-21T12:18:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T12:18:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:44.695 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-05-21T12:18:44.695 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:44.695 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:44.695 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-21 12:18:20.241448957 +0000 2024-05-21T12:18:44.696 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.696 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-21 12:14:47.306506761 +0000 2024-05-21T12:18:44.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T12:18:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-21T12:18:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-21T12:18:44.763 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000215997 s, 2.4 MB/s 2024-05-21T12:18:44.765 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T12:18:44.825 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:18:44.825 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T12:18:44.846 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T12:18:44.846 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-05-21T12:18:44.908 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T12:18:44.992 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:44.992 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-05-21T12:18:44.992 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:44.992 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:44.992 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-05-21 12:18:13.019296783 +0000 2024-05-21T12:18:44.992 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-05-21 12:14:39.060940929 +0000 2024-05-21T12:18:44.992 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-05-21 12:14:39.060940929 +0000 2024-05-21T12:18:44.993 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-05-21 12:14:39.060940929 +0000 2024-05-21T12:18:44.993 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T12:18:45.022 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-05-21T12:18:45.022 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-05-21T12:18:45.022 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000265781 s, 1.9 MB/s 2024-05-21T12:18:45.023 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T12:18:45.082 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-05-21T12:18:45.140 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T12:18:45.140 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:45.140 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-05-21T12:18:45.140 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:45.140 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:45.140 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-05-21 12:18:13.050296273 +0000 2024-05-21T12:18:45.141 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-05-21 12:14:39.061940911 +0000 2024-05-21T12:18:45.141 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-05-21 12:14:39.061940911 +0000 2024-05-21T12:18:45.141 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-05-21 12:14:39.061940911 +0000 2024-05-21T12:18:45.141 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T12:18:45.206 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-05-21T12:18:45.207 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-05-21T12:18:45.207 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.00021378 s, 2.4 MB/s 2024-05-21T12:18:45.208 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T12:18:45.265 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-05-21T12:18:45.321 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T12:18:45.321 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:45.321 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-05-21T12:18:45.321 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:45.321 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:45.322 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-05-21 12:18:13.079295795 +0000 2024-05-21T12:18:45.322 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-05-21 12:14:39.059940946 +0000 2024-05-21T12:18:45.322 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-05-21 12:14:39.059940946 +0000 2024-05-21T12:18:45.322 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-05-21 12:14:39.059940946 +0000 2024-05-21T12:18:45.322 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T12:18:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-05-21T12:18:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-05-21T12:18:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000217263 s, 2.4 MB/s 2024-05-21T12:18:45.387 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T12:18:45.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:45 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:45.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:45 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:45.445 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-05-21T12:18:45.500 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T12:18:45.501 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T12:18:45.501 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-05-21T12:18:45.501 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T12:18:45.501 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T12:18:45.501 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-05-21 12:18:13.109295301 +0000 2024-05-21T12:18:45.501 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-05-21 12:14:39.059940946 +0000 2024-05-21T12:18:45.501 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-05-21 12:14:39.059940946 +0000 2024-05-21T12:18:45.501 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-05-21 12:14:39.059940946 +0000 2024-05-21T12:18:45.501 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T12:18:45.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:45 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:45.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:45 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-05-21T12:18:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-05-21T12:18:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000258268 s, 2.0 MB/s 2024-05-21T12:18:45.565 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T12:18:45.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:45 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:45.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:45 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:18:45.622 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-05-21T12:18:45.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T12:18:45.877 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:18:46.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:46 smithi045 ceph-mon[30165]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:46.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:46 smithi072 ceph-mon[33912]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:46 smithi092 ceph-mon[33665]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:47.601 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T12:18:47.601 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T12:18:47.601 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-21T12:18:47.601 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-21T12:18:47.601 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00838624 s, 1.3 GB/s 2024-05-21T12:18:47.601 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-21T12:18:48.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-05-21T12:18:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:48 smithi045 ceph-mon[30165]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:48.459 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:18:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:48 smithi092 ceph-mon[33665]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:48.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:48 smithi072 ceph-mon[33912]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:50 smithi092 ceph-mon[33665]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:18:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:18:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:50 smithi045 ceph-mon[30165]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:50.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:18:50.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:18:50.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:50.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:50 smithi072 ceph-mon[33912]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:50.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:18:50.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:18:50.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:51.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:51 smithi045 ceph-mon[30165]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:51.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:51 smithi072 ceph-mon[33912]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:51.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:51 smithi092 ceph-mon[33665]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:18:52.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:52 smithi045 ceph-mon[30165]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:52.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:52 smithi072 ceph-mon[33912]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:52.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:52 smithi092 ceph-mon[33665]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:53.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:53 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3399933018' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a991dfbc-70b8-4c8d-8913-19b971ca07d5"} : dispatch 2024-05-21T12:18:53.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:53 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3399933018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a991dfbc-70b8-4c8d-8913-19b971ca07d5"}]': finished 2024-05-21T12:18:53.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:53 smithi045 ceph-mon[30165]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T12:18:53.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:18:53.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:53 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3399933018' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a991dfbc-70b8-4c8d-8913-19b971ca07d5"} : dispatch 2024-05-21T12:18:53.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:53 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3399933018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a991dfbc-70b8-4c8d-8913-19b971ca07d5"}]': finished 2024-05-21T12:18:53.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:53 smithi072 ceph-mon[33912]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T12:18:53.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:18:53.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:53 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3399933018' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a991dfbc-70b8-4c8d-8913-19b971ca07d5"} : dispatch 2024-05-21T12:18:53.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:53 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3399933018' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a991dfbc-70b8-4c8d-8913-19b971ca07d5"}]': finished 2024-05-21T12:18:53.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:53 smithi092 ceph-mon[33665]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T12:18:53.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:18:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:54 smithi045 ceph-mon[30165]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:54.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:54 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3563879297' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:18:54.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:54 smithi072 ceph-mon[33912]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:54.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:54 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3563879297' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:18:54.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:54 smithi092 ceph-mon[33665]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:54.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:54 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3563879297' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:18:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:56 smithi045 ceph-mon[30165]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:56.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:56 smithi072 ceph-mon[33912]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:56 smithi092 ceph-mon[33665]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:58.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:58 smithi045 ceph-mon[30165]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:58.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:58 smithi072 ceph-mon[33912]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:58 smithi092 ceph-mon[33665]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:18:59.559 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-05-21T12:18:59.559 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:18:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-05-21T12:18:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:18:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:18:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-05-21T12:18:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:18:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:00 smithi045 ceph-mon[30165]: Deploying daemon osd.0 on smithi045 2024-05-21T12:19:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:00 smithi045 ceph-mon[30165]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:00.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:00 smithi072 ceph-mon[33912]: Deploying daemon osd.0 on smithi045 2024-05-21T12:19:00.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:00 smithi072 ceph-mon[33912]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:00.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:00 smithi092 ceph-mon[33665]: Deploying daemon osd.0 on smithi045 2024-05-21T12:19:00.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:00 smithi092 ceph-mon[33665]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:02.530 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:02 smithi045 ceph-mon[30165]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:02.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:02 smithi072 ceph-mon[33912]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:02 smithi092 ceph-mon[33665]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:04 smithi045 ceph-mon[30165]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:04 smithi072 ceph-mon[33912]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:04 smithi092 ceph-mon[33665]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:05.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:05 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:05.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:05 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:05.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:05 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:05.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:05.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:05.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:05 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:05.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:05 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:05.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:05 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:05.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:05 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:06.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:06 smithi045 ceph-mon[30165]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:06 smithi092 ceph-mon[33665]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:06.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:06 smithi072 ceph-mon[33912]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:07.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:07 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:07.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:07 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:07.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:07 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:07.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:07 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:07.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:07 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:07.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:07 smithi045 ceph-mon[30165]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-05-21T12:19:07.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:07 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:07.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:07 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:07.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:07 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:07.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:07 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:07.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:07 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:07.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:07 smithi072 ceph-mon[33912]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-05-21T12:19:07.952 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-05-21T12:19:08.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:07 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:08.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:07 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:08.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:07 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:08.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:07 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:08.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:07 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:08.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:07 smithi092 ceph-mon[33665]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-05-21T12:19:08.552 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.0.service 2024-05-21T12:19:08.554 INFO:tasks.cephadm:Deploying osd.1 on smithi045 with /dev/vg_nvme/lv_3... 2024-05-21T12:19:08.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T12:19:08.811 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:19:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0[44772]: 2024-05-21T12:19:08.651+0000 7f0a5895c700 -1 osd.0 0 waiting for initial osdmap 2024-05-21T12:19:08.812 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:19:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0[44772]: 2024-05-21T12:19:08.654+0000 7f0a5474f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T12:19:08.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:08 smithi045 ceph-mon[30165]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:08.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:08 smithi045 ceph-mon[30165]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T12:19:08.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:08 smithi045 ceph-mon[30165]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T12:19:08.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:08 smithi045 ceph-mon[30165]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-05-21T12:19:08.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:08.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:08.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:08.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:08.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:08 smithi072 ceph-mon[33912]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:08.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:08 smithi072 ceph-mon[33912]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T12:19:08.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:08 smithi072 ceph-mon[33912]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T12:19:08.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:08 smithi072 ceph-mon[33912]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-05-21T12:19:08.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:08.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:08.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:08.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:09.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:08 smithi092 ceph-mon[33665]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:08 smithi092 ceph-mon[33665]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T12:19:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:08 smithi092 ceph-mon[33665]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T12:19:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:08 smithi092 ceph-mon[33665]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-05-21T12:19:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:09.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:09 smithi045 ceph-mon[30165]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T12:19:09.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:09 smithi045 ceph-mon[30165]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T12:19:09.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:09 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:09.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:09 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:09.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:09 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:09.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:09 smithi072 ceph-mon[33912]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T12:19:09.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:09 smithi072 ceph-mon[33912]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T12:19:09.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:09 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:09.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:09 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:09.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:09 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:10.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:09 smithi092 ceph-mon[33665]: from='osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T12:19:10.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:09 smithi092 ceph-mon[33665]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T12:19:10.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:09 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:10.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:09 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:10.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:09 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:19:10.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:10 smithi045 ceph-mon[30165]: purged_snaps scrub starts 2024-05-21T12:19:10.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:10 smithi045 ceph-mon[30165]: purged_snaps scrub ok 2024-05-21T12:19:10.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:10 smithi045 ceph-mon[30165]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:10.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:10 smithi045 ceph-mon[30165]: osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727] boot 2024-05-21T12:19:10.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:10 smithi045 ceph-mon[30165]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T12:19:10.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:10 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:10.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:10 smithi072 ceph-mon[33912]: purged_snaps scrub starts 2024-05-21T12:19:10.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:10 smithi072 ceph-mon[33912]: purged_snaps scrub ok 2024-05-21T12:19:10.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:10 smithi072 ceph-mon[33912]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:10.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:10 smithi072 ceph-mon[33912]: osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727] boot 2024-05-21T12:19:10.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:10 smithi072 ceph-mon[33912]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T12:19:10.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:10 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:11.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:10 smithi092 ceph-mon[33665]: purged_snaps scrub starts 2024-05-21T12:19:11.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:10 smithi092 ceph-mon[33665]: purged_snaps scrub ok 2024-05-21T12:19:11.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:10 smithi092 ceph-mon[33665]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T12:19:11.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:10 smithi092 ceph-mon[33665]: osd.0 [v2:172.21.15.45:6802/1361765727,v1:172.21.15.45:6803/1361765727] boot 2024-05-21T12:19:11.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:10 smithi092 ceph-mon[33665]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T12:19:11.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:10 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-05-21T12:19:11.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:11 smithi045 ceph-mon[30165]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T12:19:11.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:11.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:11.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:11.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:11.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:11.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:11.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:11 smithi092 ceph-mon[33665]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T12:19:12.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:12.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:12.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:12.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:11 smithi072 ceph-mon[33912]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T12:19:12.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:12.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:12.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:12.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:12.547 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T12:19:12.547 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T12:19:12.547 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-21T12:19:12.547 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-21T12:19:12.547 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00860573 s, 1.2 GB/s 2024-05-21T12:19:12.547 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-21T12:19:12.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:12 smithi045 ceph-mon[30165]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:12.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:12 smithi045 ceph-mon[30165]: Detected new or changed devices on smithi045 2024-05-21T12:19:12.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:12 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-21T12:19:13.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:12 smithi092 ceph-mon[33665]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:13.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:12 smithi092 ceph-mon[33665]: Detected new or changed devices on smithi045 2024-05-21T12:19:13.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:12 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-21T12:19:13.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-05-21T12:19:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:12 smithi072 ceph-mon[33912]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:12 smithi072 ceph-mon[33912]: Detected new or changed devices on smithi045 2024-05-21T12:19:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:12 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-21T12:19:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:19:14.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:14 smithi045 ceph-mon[30165]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:15.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:14 smithi092 ceph-mon[33665]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:15.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:14 smithi072 ceph-mon[33912]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:15.931 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:15 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:19:15.931 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:15 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:19:15.931 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:15 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:16.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:15 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:19:16.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:15 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:19:16.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:15 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:16.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:15 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:19:16.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:15 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:19:16.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:15 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:16.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:16 smithi045 ceph-mon[30165]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:19:16.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:16 smithi045 ceph-mon[30165]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:17.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:16 smithi092 ceph-mon[33665]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:19:17.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:16 smithi092 ceph-mon[33665]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:17.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:16 smithi072 ceph-mon[33912]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:19:17.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:16 smithi072 ceph-mon[33912]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:18.968 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:18 smithi072 ceph-mon[33912]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:18.968 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:18 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1424494244' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9ae507f9-6943-42d8-b26e-af49f064dc74"} : dispatch 2024-05-21T12:19:18.968 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:18 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1424494244' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ae507f9-6943-42d8-b26e-af49f064dc74"}]': finished 2024-05-21T12:19:18.969 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:18 smithi072 ceph-mon[33912]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T12:19:18.969 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:18 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:18.969 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:18 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3942809653' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:19:19.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:18 smithi092 ceph-mon[33665]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:19.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:18 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1424494244' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9ae507f9-6943-42d8-b26e-af49f064dc74"} : dispatch 2024-05-21T12:19:19.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:18 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1424494244' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ae507f9-6943-42d8-b26e-af49f064dc74"}]': finished 2024-05-21T12:19:19.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:18 smithi092 ceph-mon[33665]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T12:19:19.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:18 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:19.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:18 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3942809653' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:19:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:18 smithi045 ceph-mon[30165]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:18 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1424494244' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9ae507f9-6943-42d8-b26e-af49f064dc74"} : dispatch 2024-05-21T12:19:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:18 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1424494244' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ae507f9-6943-42d8-b26e-af49f064dc74"}]': finished 2024-05-21T12:19:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:18 smithi045 ceph-mon[30165]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T12:19:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:18 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:18 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3942809653' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:19:21.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:20 smithi092 ceph-mon[33665]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:21.021 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:20 smithi045 ceph-mon[30165]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:21.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:20 smithi072 ceph-mon[33912]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:22.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:22 smithi045 ceph-mon[30165]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:23.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:22 smithi092 ceph-mon[33665]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:23.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:22 smithi072 ceph-mon[33912]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:24.971 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:24 smithi045 ceph-mon[30165]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:24.972 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:24 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-05-21T12:19:24.972 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:24 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:25.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:24 smithi092 ceph-mon[33665]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:25.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:24 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-05-21T12:19:25.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:24 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:25.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:24 smithi072 ceph-mon[33912]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:25.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:24 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-05-21T12:19:25.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:24 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:25.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:25 smithi045 ceph-mon[30165]: Deploying daemon osd.1 on smithi045 2024-05-21T12:19:26.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:25 smithi092 ceph-mon[33665]: Deploying daemon osd.1 on smithi045 2024-05-21T12:19:26.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:25 smithi072 ceph-mon[33912]: Deploying daemon osd.1 on smithi045 2024-05-21T12:19:26.809 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:26 smithi045 ceph-mon[30165]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:27.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:26 smithi092 ceph-mon[33665]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:27.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:26 smithi072 ceph-mon[33912]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:28.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:28 smithi045 ceph-mon[30165]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:29.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:28 smithi092 ceph-mon[33665]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:29.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:28 smithi072 ceph-mon[33912]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:30 smithi072 ceph-mon[33912]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:30 smithi092 ceph-mon[33665]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:31.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:30 smithi045 ceph-mon[30165]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:31.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:31.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:31.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:32 smithi072 ceph-mon[33912]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:32.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:32.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:32.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:32.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:32.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:32.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:32 smithi045 ceph-mon[30165]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:32.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:32.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:32.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:32.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:32.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:33.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:32 smithi092 ceph-mon[33665]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:33.333 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 1 on host 'smithi045' 2024-05-21T12:19:33.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:33 smithi072 ceph-mon[33912]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-05-21T12:19:33.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:33 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:33.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:33 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:33.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:33 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:33.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:33 smithi045 ceph-mon[30165]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-05-21T12:19:33.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:33 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:33.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:33 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:33.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:33 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:34.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:33 smithi092 ceph-mon[33665]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-05-21T12:19:34.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:33 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:34.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:33 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:34.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:33 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:34.007 DEBUG:teuthology.orchestra.run.smithi045:osd.1> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.1.service 2024-05-21T12:19:34.009 INFO:tasks.cephadm:Deploying osd.2 on smithi072 with /dev/vg_nvme/lv_4... 2024-05-21T12:19:34.009 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T12:19:34.253 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T12:19:34.659 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:34 smithi072 ceph-mon[33912]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:34.659 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:34 smithi072 ceph-mon[33912]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T12:19:34.660 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:34 smithi072 ceph-mon[33912]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T12:19:34.660 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:34 smithi072 ceph-mon[33912]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-05-21T12:19:34.660 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:34.862 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:34 smithi045 ceph-mon[30165]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:34.862 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:34 smithi045 ceph-mon[30165]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T12:19:34.863 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:34 smithi045 ceph-mon[30165]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T12:19:34.863 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:34 smithi045 ceph-mon[30165]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-05-21T12:19:34.863 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:34.863 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:19:34 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1[50681]: 2024-05-21T12:19:34.602+0000 7fd0ee5bc700 -1 osd.1 0 waiting for initial osdmap 2024-05-21T12:19:34.863 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:19:34 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1[50681]: 2024-05-21T12:19:34.606+0000 7fd0ea3af700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T12:19:35.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:34 smithi092 ceph-mon[33665]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:35.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:34 smithi092 ceph-mon[33665]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T12:19:35.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:34 smithi092 ceph-mon[33665]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T12:19:35.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:34 smithi092 ceph-mon[33665]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]} : dispatch 2024-05-21T12:19:35.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:35.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:35 smithi045 ceph-mon[30165]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T12:19:35.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:35 smithi045 ceph-mon[30165]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T12:19:35.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:35.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:35 smithi072 ceph-mon[33912]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T12:19:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:35 smithi072 ceph-mon[33912]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T12:19:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:35.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:36.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:35 smithi092 ceph-mon[33665]: from='osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-21T12:19:36.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:35 smithi092 ceph-mon[33665]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T12:19:36.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:36.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:36.159 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T12:19:36.159 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T12:19:36.159 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-21T12:19:36.159 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-21T12:19:36.160 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114466 s, 916 MB/s 2024-05-21T12:19:36.160 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 2024-05-21T12:19:36.160 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-21T12:19:36.731 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:36 smithi072 ceph-mon[33912]: purged_snaps scrub starts 2024-05-21T12:19:36.732 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:36 smithi072 ceph-mon[33912]: purged_snaps scrub ok 2024-05-21T12:19:36.732 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:36 smithi072 ceph-mon[33912]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:36.732 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:36 smithi072 ceph-mon[33912]: osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365] boot 2024-05-21T12:19:36.732 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:36 smithi072 ceph-mon[33912]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T12:19:36.732 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:36.761 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_4 2024-05-21T12:19:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:36 smithi045 ceph-mon[30165]: purged_snaps scrub starts 2024-05-21T12:19:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:36 smithi045 ceph-mon[30165]: purged_snaps scrub ok 2024-05-21T12:19:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:36 smithi045 ceph-mon[30165]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:36 smithi045 ceph-mon[30165]: osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365] boot 2024-05-21T12:19:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:36 smithi045 ceph-mon[30165]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T12:19:36.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:37.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:36 smithi092 ceph-mon[33665]: purged_snaps scrub starts 2024-05-21T12:19:37.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:36 smithi092 ceph-mon[33665]: purged_snaps scrub ok 2024-05-21T12:19:37.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:36 smithi092 ceph-mon[33665]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T12:19:37.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:36 smithi092 ceph-mon[33665]: osd.1 [v2:172.21.15.45:6810/2489398365,v1:172.21.15.45:6811/2489398365] boot 2024-05-21T12:19:37.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:36 smithi092 ceph-mon[33665]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T12:19:37.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-05-21T12:19:37.004 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T12:19:37.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T12:19:37.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:37.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:37.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:37.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:37.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:37.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:37.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:37.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T12:19:38.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:38.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:38.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:38.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:38.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T12:19:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-05-21T12:19:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:38.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:38 smithi072 ceph-mon[33912]: Detected new or changed devices on smithi045 2024-05-21T12:19:38.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:38 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-21T12:19:38.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:38 smithi072 ceph-mon[33912]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:39.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:38 smithi092 ceph-mon[33665]: Detected new or changed devices on smithi045 2024-05-21T12:19:39.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:38 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-21T12:19:39.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:38 smithi092 ceph-mon[33665]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:38 smithi045 ceph-mon[30165]: Detected new or changed devices on smithi045 2024-05-21T12:19:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:38 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-21T12:19:39.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:38 smithi045 ceph-mon[30165]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:39.855 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:39 smithi072 ceph-mon[33912]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:19:39.856 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:19:39.856 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:19:39.856 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:40.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:39 smithi092 ceph-mon[33665]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:19:40.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:19:40.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:19:40.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:40.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:39 smithi045 ceph-mon[30165]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:19:40.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:19:40.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:19:40.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:40.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:40 smithi072 ceph-mon[33912]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:41.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:40 smithi092 ceph-mon[33665]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:41.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:40 smithi045 ceph-mon[30165]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:41.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:41 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3872331259' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"} : dispatch 2024-05-21T12:19:41.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:41 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"} : dispatch 2024-05-21T12:19:41.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:41 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"}]': finished 2024-05-21T12:19:41.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:41 smithi072 ceph-mon[33912]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T12:19:41.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:41 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3872331259' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"} : dispatch 2024-05-21T12:19:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:41 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"} : dispatch 2024-05-21T12:19:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:41 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"}]': finished 2024-05-21T12:19:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:41 smithi092 ceph-mon[33665]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T12:19:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:42.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:41 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3872331259' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"} : dispatch 2024-05-21T12:19:42.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:41 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"} : dispatch 2024-05-21T12:19:42.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:41 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a520290-21ec-4b31-9e77-9d71cb659de6"}]': finished 2024-05-21T12:19:42.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:41 smithi045 ceph-mon[30165]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T12:19:42.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:42.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:42 smithi072 ceph-mon[33912]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:42.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:42 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/4229206666' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:19:43.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:42 smithi092 ceph-mon[33665]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:43.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:42 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/4229206666' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:19:43.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:42 smithi045 ceph-mon[30165]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:43.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:42 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/4229206666' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:19:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:44 smithi072 ceph-mon[33912]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:45.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:44 smithi092 ceph-mon[33665]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:45.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:44 smithi045 ceph-mon[30165]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:46.897 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:46 smithi072 ceph-mon[33912]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:47.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:46 smithi092 ceph-mon[33665]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:47.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:46 smithi045 ceph-mon[30165]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:48.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:48 smithi072 ceph-mon[33912]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:48.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-05-21T12:19:48.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:48.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:48 smithi072 ceph-mon[33912]: Deploying daemon osd.2 on smithi072 2024-05-21T12:19:49.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:48 smithi092 ceph-mon[33665]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:49.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-05-21T12:19:49.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:49.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:48 smithi092 ceph-mon[33665]: Deploying daemon osd.2 on smithi072 2024-05-21T12:19:49.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:48 smithi045 ceph-mon[30165]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:49.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-05-21T12:19:49.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:49.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:48 smithi045 ceph-mon[30165]: Deploying daemon osd.2 on smithi072 2024-05-21T12:19:50.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:50 smithi045 ceph-mon[30165]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:50.894 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:50 smithi072 ceph-mon[33912]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:51.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:50 smithi092 ceph-mon[33665]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:52.874 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:52 smithi072 ceph-mon[33912]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:53.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:52 smithi092 ceph-mon[33665]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:53.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:52 smithi045 ceph-mon[30165]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:54.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:54.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:54.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:54.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:53 smithi092 ceph-mon[33665]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:54.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:54.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:54.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:54.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:53 smithi045 ceph-mon[30165]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:54.337 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:54.337 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:54.337 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:54.337 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:53 smithi072 ceph-mon[33912]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:56.040 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.041 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.041 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:56.041 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:56.041 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:56.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:56.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:19:56.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:19:56.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:56.462 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 2 on host 'smithi072' 2024-05-21T12:19:57.043 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:56 smithi072 ceph-mon[33912]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:57.043 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:56 smithi072 ceph-mon[33912]: from='osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-05-21T12:19:57.043 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:56 smithi072 ceph-mon[33912]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-05-21T12:19:57.043 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:56 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:57.043 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:56 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:57.043 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:56 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:57.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:56 smithi045 ceph-mon[30165]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:57.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:56 smithi045 ceph-mon[30165]: from='osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-05-21T12:19:57.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:56 smithi045 ceph-mon[30165]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-05-21T12:19:57.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:56 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:57.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:56 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:57.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:56 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:57.072 DEBUG:teuthology.orchestra.run.smithi072:osd.2> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.2.service 2024-05-21T12:19:57.074 INFO:tasks.cephadm:Deploying osd.3 on smithi072 with /dev/vg_nvme/lv_3... 2024-05-21T12:19:57.075 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T12:19:57.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:56 smithi092 ceph-mon[33665]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:57.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:56 smithi092 ceph-mon[33665]: from='osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-05-21T12:19:57.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:56 smithi092 ceph-mon[33665]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-05-21T12:19:57.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:56 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:57.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:56 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:19:57.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:56 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:19:58.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:57 smithi045 ceph-mon[30165]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T12:19:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:57 smithi045 ceph-mon[30165]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T12:19:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:57 smithi045 ceph-mon[30165]: from='osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:19:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:57 smithi045 ceph-mon[30165]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:19:58.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:57 smithi072 ceph-mon[33912]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T12:19:58.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:57 smithi072 ceph-mon[33912]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T12:19:58.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:58.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:57 smithi072 ceph-mon[33912]: from='osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:19:58.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:57 smithi072 ceph-mon[33912]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:19:58.167 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:19:57 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2[37514]: 2024-05-21T12:19:57.781+0000 7fcd0a630700 -1 osd.2 0 waiting for initial osdmap 2024-05-21T12:19:58.167 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:19:57 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2[37514]: 2024-05-21T12:19:57.784+0000 7fcd06423700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T12:19:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:57 smithi092 ceph-mon[33665]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T12:19:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:57 smithi092 ceph-mon[33665]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T12:19:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:57 smithi092 ceph-mon[33665]: from='osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:19:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:57 smithi092 ceph-mon[33665]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:19:58.424 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T12:19:59.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:58 smithi045 ceph-mon[30165]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:59.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:58 smithi045 ceph-mon[30165]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:19:59.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:58 smithi045 ceph-mon[30165]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T12:19:59.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:59.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:59.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:58 smithi072 ceph-mon[33912]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:59.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:58 smithi072 ceph-mon[33912]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:19:59.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:58 smithi072 ceph-mon[33912]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T12:19:59.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:59.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:59.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:58 smithi092 ceph-mon[33665]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T12:19:59.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:58 smithi092 ceph-mon[33665]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:19:59.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:58 smithi092 ceph-mon[33665]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T12:19:59.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:59.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:59.870 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:59 smithi072 ceph-mon[33912]: purged_snaps scrub starts 2024-05-21T12:19:59.870 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:59 smithi072 ceph-mon[33912]: purged_snaps scrub ok 2024-05-21T12:19:59.870 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:59 smithi072 ceph-mon[33912]: osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895] boot 2024-05-21T12:19:59.870 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:59 smithi072 ceph-mon[33912]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T12:19:59.870 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:19:59.870 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:19:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-05-21T12:20:00.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:59 smithi045 ceph-mon[30165]: purged_snaps scrub starts 2024-05-21T12:20:00.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:59 smithi045 ceph-mon[30165]: purged_snaps scrub ok 2024-05-21T12:20:00.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:59 smithi045 ceph-mon[30165]: osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895] boot 2024-05-21T12:20:00.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:59 smithi045 ceph-mon[30165]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T12:20:00.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:20:00.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:19:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-05-21T12:20:00.252 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T12:20:00.252 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T12:20:00.252 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-21T12:20:00.253 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-21T12:20:00.253 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137184 s, 764 MB/s 2024-05-21T12:20:00.253 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 2024-05-21T12:20:00.253 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-21T12:20:00.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:59 smithi092 ceph-mon[33665]: purged_snaps scrub starts 2024-05-21T12:20:00.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:59 smithi092 ceph-mon[33665]: purged_snaps scrub ok 2024-05-21T12:20:00.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:59 smithi092 ceph-mon[33665]: osd.2 [v2:172.21.15.72:6800/630200895,v1:172.21.15.72:6801/630200895] boot 2024-05-21T12:20:00.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:59 smithi092 ceph-mon[33665]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T12:20:00.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-05-21T12:20:00.254 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:19:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-05-21T12:20:00.851 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_3 2024-05-21T12:20:00.875 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:00 smithi072 ceph-mon[33912]: pgmap v84: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:00.875 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:00 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T12:20:00.875 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:00 smithi072 ceph-mon[33912]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T12:20:00.876 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:00 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-05-21T12:20:00.876 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:00 smithi072 ceph-mon[33912]: overall HEALTH_OK 2024-05-21T12:20:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:00 smithi045 ceph-mon[30165]: pgmap v84: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:00 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T12:20:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:00 smithi045 ceph-mon[30165]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T12:20:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:00 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-05-21T12:20:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:00 smithi045 ceph-mon[30165]: overall HEALTH_OK 2024-05-21T12:20:01.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:00 smithi092 ceph-mon[33665]: pgmap v84: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:01.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:00 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T12:20:01.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:00 smithi092 ceph-mon[33665]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T12:20:01.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:00 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-05-21T12:20:01.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:00 smithi092 ceph-mon[33665]: overall HEALTH_OK 2024-05-21T12:20:01.425 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T12:20:01.800 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54432]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T12:20:01.800 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54432]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.800 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54432]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54432]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54435]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54435]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54435]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54438]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54438]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54438]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54438]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.801 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54441]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T12:20:01.802 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54441]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.802 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54441]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.802 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54441]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54445]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T12:20:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54445]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54445]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54445]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54449]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T12:20:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54449]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54449]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 sudo[54449]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.803 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40984]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T12:20:01.803 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40984]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.803 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40984]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.803 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40984]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.803 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T12:20:01.803 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.803 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40987]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.804 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40987]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.804 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40974]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T12:20:01.804 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40974]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.804 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40974]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.804 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40974]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:01.804 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40977]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T12:20:01.804 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:01.804 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40977]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:01.804 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:20:01 smithi072 sudo[40977]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 sudo[35401]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T12:20:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 sudo[35401]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 sudo[35401]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 sudo[35401]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 sudo[35404]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 sudo[35404]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 sudo[35404]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 sudo[35404]: pam_unix(sudo:session): session closed for user root 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: Detected new or changed devices on smithi072 2024-05-21T12:20:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi072 to 21184M 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.005 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.007 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:01 smithi092 ceph-mon[33665]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T12:20:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T12:20:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: Detected new or changed devices on smithi072 2024-05-21T12:20:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi072 to 21184M 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:02.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:01 smithi045 ceph-mon[30165]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:02.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: Detected new or changed devices on smithi072 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi072 to 21184M 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:02.168 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:02.169 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T12:20:02.170 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-05-21T12:20:02.170 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-05-21T12:20:02.170 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-05-21T12:20:02.170 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:01 smithi072 ceph-mon[33912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T12:20:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:03 smithi072 ceph-mon[33912]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T12:20:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:03.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:03 smithi092 ceph-mon[33665]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T12:20:03.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:03.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:03.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:03.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:03 smithi045 ceph-mon[30165]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T12:20:03.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:03.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:03.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:04 smithi072 ceph-mon[33912]: from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:04 smithi072 ceph-mon[33912]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:04 smithi072 ceph-mon[33912]: mgrmap e14: x(active, since 2m) 2024-05-21T12:20:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:04 smithi092 ceph-mon[33665]: from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:04.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:04 smithi092 ceph-mon[33665]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:04.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:04 smithi092 ceph-mon[33665]: mgrmap e14: x(active, since 2m) 2024-05-21T12:20:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:04 smithi045 ceph-mon[30165]: from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:04.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:04 smithi045 ceph-mon[30165]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:04.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:04 smithi045 ceph-mon[30165]: mgrmap e14: x(active, since 2m) 2024-05-21T12:20:06.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:06 smithi072 ceph-mon[33912]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:06.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:06 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3636659914' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"} : dispatch 2024-05-21T12:20:06.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:06 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"} : dispatch 2024-05-21T12:20:06.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:06 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"}]': finished 2024-05-21T12:20:06.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:06 smithi072 ceph-mon[33912]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T12:20:06.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:06 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:06 smithi092 ceph-mon[33665]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:06 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3636659914' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"} : dispatch 2024-05-21T12:20:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:06 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"} : dispatch 2024-05-21T12:20:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:06 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"}]': finished 2024-05-21T12:20:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:06 smithi092 ceph-mon[33665]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T12:20:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:06 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:06 smithi045 ceph-mon[30165]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:06.849 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:06 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3636659914' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"} : dispatch 2024-05-21T12:20:06.849 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:06 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"} : dispatch 2024-05-21T12:20:06.849 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:06 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5006e39c-d6e6-44a6-8e2f-0501b36aa3b2"}]': finished 2024-05-21T12:20:06.849 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:06 smithi045 ceph-mon[30165]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T12:20:06.849 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:06 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:07.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:07 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/4294103737' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:07.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:07 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/4294103737' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:07.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:07 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/4294103737' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:08.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:08 smithi072 ceph-mon[33912]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:08 smithi092 ceph-mon[33665]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:08 smithi045 ceph-mon[30165]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:10.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:10 smithi072 ceph-mon[33912]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:10.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:10 smithi092 ceph-mon[33665]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:10 smithi045 ceph-mon[30165]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:12.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:12 smithi072 ceph-mon[33912]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:12.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:12 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-05-21T12:20:12.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:12 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:12 smithi092 ceph-mon[33665]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:12.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:12 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-05-21T12:20:12.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:12 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:12.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:12 smithi045 ceph-mon[30165]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:12.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:12 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-05-21T12:20:12.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:12 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:13.647 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:13 smithi072 ceph-mon[33912]: Deploying daemon osd.3 on smithi072 2024-05-21T12:20:13.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:13 smithi092 ceph-mon[33665]: Deploying daemon osd.3 on smithi072 2024-05-21T12:20:13.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:13 smithi045 ceph-mon[30165]: Deploying daemon osd.3 on smithi072 2024-05-21T12:20:14.555 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:14 smithi072 ceph-mon[33912]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:14 smithi092 ceph-mon[33665]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:14 smithi045 ceph-mon[30165]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:16.669 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:16 smithi072 ceph-mon[33912]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:16 smithi092 ceph-mon[33665]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:16.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:16 smithi045 ceph-mon[30165]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:18.980 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:18 smithi072 ceph-mon[33912]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:18.980 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:18 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:18.980 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:18 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:18.981 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:18 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:19.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:18 smithi045 ceph-mon[30165]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:18 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:18 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:18 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:19.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:18 smithi092 ceph-mon[33665]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:19.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:18 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:19.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:18 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:19.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:18 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:21.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:20 smithi045 ceph-mon[30165]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:21.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:20 smithi072 ceph-mon[33912]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:21.192 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 3 on host 'smithi072' 2024-05-21T12:20:21.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:20 smithi092 ceph-mon[33665]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:21.775 DEBUG:teuthology.orchestra.run.smithi072:osd.3> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.3.service 2024-05-21T12:20:21.817 INFO:tasks.cephadm:Deploying osd.4 on smithi072 with /dev/vg_nvme/lv_2... 2024-05-21T12:20:21.818 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T12:20:22.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:21 smithi045 ceph-mon[30165]: from='osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-05-21T12:20:22.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:21 smithi045 ceph-mon[30165]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-05-21T12:20:22.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:21 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:21 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:21 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:21 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.095 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T12:20:22.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:21 smithi072 ceph-mon[33912]: from='osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-05-21T12:20:22.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:21 smithi072 ceph-mon[33912]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-05-21T12:20:22.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:21 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:21 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:21 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:21 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:21 smithi092 ceph-mon[33665]: from='osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-05-21T12:20:22.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:21 smithi092 ceph-mon[33665]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-05-21T12:20:22.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:22.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:21 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:23.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:22 smithi045 ceph-mon[30165]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:23.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:22 smithi045 ceph-mon[30165]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T12:20:23.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:22 smithi045 ceph-mon[30165]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T12:20:23.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:22 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:23.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:22 smithi045 ceph-mon[30165]: from='osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:23.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:22 smithi045 ceph-mon[30165]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:23.145 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:22 smithi072 ceph-mon[33912]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:23.145 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:22 smithi072 ceph-mon[33912]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T12:20:23.145 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:22 smithi072 ceph-mon[33912]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T12:20:23.145 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:22 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:23.145 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:22 smithi072 ceph-mon[33912]: from='osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:23.145 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:22 smithi072 ceph-mon[33912]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:22 smithi092 ceph-mon[33665]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:22 smithi092 ceph-mon[33665]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T12:20:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:22 smithi092 ceph-mon[33665]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T12:20:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:22 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:23.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:22 smithi092 ceph-mon[33665]: from='osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:23.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:22 smithi092 ceph-mon[33665]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:23.416 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:20:23 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3[42945]: 2024-05-21T12:20:23.214+0000 7f89156e0700 -1 osd.3 0 waiting for initial osdmap 2024-05-21T12:20:23.416 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:20:23 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3[42945]: 2024-05-21T12:20:23.220+0000 7f890d4cb700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T12:20:24.003 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T12:20:24.003 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T12:20:24.003 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-21T12:20:24.003 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-21T12:20:24.003 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0179767 s, 583 MB/s 2024-05-21T12:20:24.003 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-21T12:20:24.066 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:23 smithi072 ceph-mon[33912]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:20:24.066 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:23 smithi072 ceph-mon[33912]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T12:20:24.066 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:23 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:23 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:23 smithi072 ceph-mon[33912]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:23 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:23 smithi072 ceph-mon[33912]: from='osd.3 ' entity='osd.3' 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:23 smithi045 ceph-mon[30165]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:23 smithi045 ceph-mon[30165]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:23 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:23 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:23 smithi045 ceph-mon[30165]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T12:20:24.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:23 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:23 smithi045 ceph-mon[30165]: from='osd.3 ' entity='osd.3' 2024-05-21T12:20:24.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:23 smithi092 ceph-mon[33665]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:20:24.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:23 smithi092 ceph-mon[33665]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T12:20:24.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:23 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:23 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:23 smithi092 ceph-mon[33665]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T12:20:24.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:23 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:24.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:23 smithi092 ceph-mon[33665]: from='osd.3 ' entity='osd.3' 2024-05-21T12:20:25.040 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:24 smithi072 ceph-mon[33912]: purged_snaps scrub starts 2024-05-21T12:20:25.040 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:24 smithi072 ceph-mon[33912]: purged_snaps scrub ok 2024-05-21T12:20:25.040 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:24 smithi072 ceph-mon[33912]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:25.040 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:24 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:25.040 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:24 smithi072 ceph-mon[33912]: osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753] boot 2024-05-21T12:20:25.040 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:24 smithi072 ceph-mon[33912]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T12:20:25.040 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:24 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:25.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:24 smithi045 ceph-mon[30165]: purged_snaps scrub starts 2024-05-21T12:20:25.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:24 smithi045 ceph-mon[30165]: purged_snaps scrub ok 2024-05-21T12:20:25.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:24 smithi045 ceph-mon[30165]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:25.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:24 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:25.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:24 smithi045 ceph-mon[30165]: osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753] boot 2024-05-21T12:20:25.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:24 smithi045 ceph-mon[30165]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T12:20:25.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:24 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:25.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:24 smithi092 ceph-mon[33665]: purged_snaps scrub starts 2024-05-21T12:20:25.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:24 smithi092 ceph-mon[33665]: purged_snaps scrub ok 2024-05-21T12:20:25.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:24 smithi092 ceph-mon[33665]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-21T12:20:25.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:24 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:25.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:24 smithi092 ceph-mon[33665]: osd.3 [v2:172.21.15.72:6808/3263314753,v1:172.21.15.72:6809/3263314753] boot 2024-05-21T12:20:25.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:24 smithi092 ceph-mon[33665]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T12:20:25.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:24 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-05-21T12:20:25.362 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_2 2024-05-21T12:20:25.812 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T12:20:26.197 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:25 smithi072 ceph-mon[33912]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T12:20:26.197 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:25 smithi072 ceph-mon[33912]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:26.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:25 smithi092 ceph-mon[33665]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T12:20:26.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:25 smithi092 ceph-mon[33665]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:26.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:25 smithi045 ceph-mon[30165]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T12:20:26.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:25 smithi045 ceph-mon[30165]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:27.051 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:26 smithi072 ceph-mon[33912]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T12:20:27.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:26 smithi092 ceph-mon[33665]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T12:20:27.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:26 smithi045 ceph-mon[30165]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T12:20:28.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: Detected new or changed devices on smithi072 2024-05-21T12:20:28.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi072 to 10592M 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:28.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:28.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:28 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: Detected new or changed devices on smithi072 2024-05-21T12:20:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi072 to 10592M 2024-05-21T12:20:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:28.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:28.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:28.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:28.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:28.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:28 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: Detected new or changed devices on smithi072 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi072 to 10592M 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:28.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:28.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:28.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:28.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:28.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:28.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:28 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:29.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:29 smithi072 ceph-mon[33912]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:29.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:29 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:29.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:29 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:29.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:29 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:29.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:29 smithi092 ceph-mon[33665]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:29.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:29 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:29.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:29 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:29.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:29 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:29.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:29 smithi045 ceph-mon[30165]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:29.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:29 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:29.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:29 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:29.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:29 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:30.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:30 smithi072 ceph-mon[33912]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:30.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:30 smithi092 ceph-mon[33665]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:30.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:30 smithi045 ceph-mon[30165]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:31.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:31 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/738121353' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"} : dispatch 2024-05-21T12:20:31.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:31 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"} : dispatch 2024-05-21T12:20:31.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:31 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"}]': finished 2024-05-21T12:20:31.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:31 smithi072 ceph-mon[33912]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T12:20:31.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:31 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:31.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:31 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/738121353' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"} : dispatch 2024-05-21T12:20:31.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:31 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"} : dispatch 2024-05-21T12:20:31.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:31 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"}]': finished 2024-05-21T12:20:31.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:31 smithi092 ceph-mon[33665]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T12:20:31.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:31 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:31.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:31 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/738121353' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"} : dispatch 2024-05-21T12:20:31.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:31 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"} : dispatch 2024-05-21T12:20:31.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:31 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "188f0178-16be-41fa-8f84-d75443654c4b"}]': finished 2024-05-21T12:20:31.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:31 smithi045 ceph-mon[30165]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T12:20:31.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:31 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:32.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:32 smithi072 ceph-mon[33912]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-21T12:20:32.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:32 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/981337260' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:32.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:32 smithi092 ceph-mon[33665]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-21T12:20:32.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:32 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/981337260' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:32.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:32 smithi045 ceph-mon[30165]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-21T12:20:32.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:32 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/981337260' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:34.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:34 smithi072 ceph-mon[33912]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T12:20:34.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:34 smithi045 ceph-mon[30165]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T12:20:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:34 smithi092 ceph-mon[33665]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T12:20:36.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:36 smithi045 ceph-mon[30165]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-21T12:20:36.626 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:36 smithi072 ceph-mon[33912]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-21T12:20:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:36 smithi092 ceph-mon[33665]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-21T12:20:38.411 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:38 smithi072 ceph-mon[33912]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T12:20:38.411 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-05-21T12:20:38.411 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:38.411 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:38 smithi072 ceph-mon[33912]: Deploying daemon osd.4 on smithi072 2024-05-21T12:20:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:38 smithi045 ceph-mon[30165]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T12:20:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-05-21T12:20:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:38 smithi045 ceph-mon[30165]: Deploying daemon osd.4 on smithi072 2024-05-21T12:20:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:38 smithi092 ceph-mon[33665]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T12:20:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-05-21T12:20:38.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:38.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:38 smithi092 ceph-mon[33665]: Deploying daemon osd.4 on smithi072 2024-05-21T12:20:40.541 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:40 smithi072 ceph-mon[33912]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T12:20:40.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:40 smithi045 ceph-mon[30165]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T12:20:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:40 smithi092 ceph-mon[33665]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T12:20:42.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:42 smithi072 ceph-mon[33912]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:42.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:42 smithi045 ceph-mon[30165]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:42 smithi092 ceph-mon[33665]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:44.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:44.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:44.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:44 smithi045 ceph-mon[30165]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:44.328 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:44.328 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:44.328 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:44.328 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:44 smithi072 ceph-mon[33912]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:44.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:44.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:44.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:44.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:44 smithi092 ceph-mon[33665]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:46.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:46 smithi072 ceph-mon[33912]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:46.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:46 smithi072 ceph-mon[33912]: from='osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-05-21T12:20:46.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:46 smithi072 ceph-mon[33912]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-05-21T12:20:46.467 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 4 on host 'smithi072' 2024-05-21T12:20:46.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:46 smithi045 ceph-mon[30165]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:46.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:46 smithi045 ceph-mon[30165]: from='osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-05-21T12:20:46.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:46 smithi045 ceph-mon[30165]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-05-21T12:20:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:46 smithi092 ceph-mon[33665]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:46 smithi092 ceph-mon[33665]: from='osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-05-21T12:20:46.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:46 smithi092 ceph-mon[33665]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-05-21T12:20:47.098 DEBUG:teuthology.orchestra.run.smithi072:osd.4> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.4.service 2024-05-21T12:20:47.100 INFO:tasks.cephadm:Deploying osd.5 on smithi092 with /dev/vg_nvme/lv_4... 2024-05-21T12:20:47.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T12:20:47.353 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: from='osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.358 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:47 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: from='osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.475 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:47 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T12:20:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T12:20:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: from='osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]} : dispatch 2024-05-21T12:20:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:47 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:48.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:20:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:20:47.703+0000 7fb39135a700 -1 osd.4 0 waiting for initial osdmap 2024-05-21T12:20:48.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:20:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:20:47.708+0000 7fb389145700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T12:20:48.423 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:48 smithi072 ceph-mon[33912]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:48.423 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:48 smithi072 ceph-mon[33912]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:20:48.423 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:48 smithi072 ceph-mon[33912]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T12:20:48.423 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:48.424 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:48.424 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:48 smithi072 ceph-mon[33912]: from='osd.4 ' entity='osd.4' 2024-05-21T12:20:48.424 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:48.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:48 smithi045 ceph-mon[30165]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:48.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:48 smithi045 ceph-mon[30165]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:20:48.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:48 smithi045 ceph-mon[30165]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T12:20:48.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:48.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:48.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:48 smithi045 ceph-mon[30165]: from='osd.4 ' entity='osd.4' 2024-05-21T12:20:48.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:48.646 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:48 smithi092 ceph-mon[33665]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:48.646 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:48 smithi092 ceph-mon[33665]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-21T12:20:48.646 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:48 smithi092 ceph-mon[33665]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T12:20:48.646 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:48.646 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:48.646 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:48 smithi092 ceph-mon[33665]: from='osd.4 ' entity='osd.4' 2024-05-21T12:20:48.646 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:49.489 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T12:20:49.489 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T12:20:49.489 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-21T12:20:49.489 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-21T12:20:49.489 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113676 s, 922 MB/s 2024-05-21T12:20:49.489 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-21T12:20:49.844 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:49 smithi092 ceph-mon[33665]: purged_snaps scrub starts 2024-05-21T12:20:49.844 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:49 smithi092 ceph-mon[33665]: purged_snaps scrub ok 2024-05-21T12:20:49.844 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:49 smithi092 ceph-mon[33665]: osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302] boot 2024-05-21T12:20:49.844 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:49 smithi092 ceph-mon[33665]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T12:20:49.844 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:49 smithi072 ceph-mon[33912]: purged_snaps scrub starts 2024-05-21T12:20:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:49 smithi072 ceph-mon[33912]: purged_snaps scrub ok 2024-05-21T12:20:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:49 smithi072 ceph-mon[33912]: osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302] boot 2024-05-21T12:20:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:49 smithi072 ceph-mon[33912]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T12:20:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:50.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:49 smithi045 ceph-mon[30165]: purged_snaps scrub starts 2024-05-21T12:20:50.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:49 smithi045 ceph-mon[30165]: purged_snaps scrub ok 2024-05-21T12:20:50.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:49 smithi045 ceph-mon[30165]: osd.4 [v2:172.21.15.72:6816/930805302,v1:172.21.15.72:6817/930805302] boot 2024-05-21T12:20:50.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:49 smithi045 ceph-mon[30165]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T12:20:50.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-05-21T12:20:50.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-05-21T12:20:50.398 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:50.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:50.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:50.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:50.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:50.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T12:20:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:50.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:50.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:50.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:50.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:50.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:50.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:50.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:51.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-21T12:20:51.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T12:20:51.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:51.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:20:51.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:20:51.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:51 smithi092 ceph-mon[33665]: Detected new or changed devices on smithi072 2024-05-21T12:20:51.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:51 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi072 to 7061M 2024-05-21T12:20:52.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:51 smithi045 ceph-mon[30165]: Detected new or changed devices on smithi072 2024-05-21T12:20:52.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:51 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi072 to 7061M 2024-05-21T12:20:52.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:51 smithi072 ceph-mon[33912]: Detected new or changed devices on smithi072 2024-05-21T12:20:52.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:51 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi072 to 7061M 2024-05-21T12:20:52.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:52 smithi092 ceph-mon[33665]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:52.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:52 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:52.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:52 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:52.807 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:52 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:53.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:52 smithi045 ceph-mon[30165]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:53.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:52 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:53.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:52 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:53.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:52 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:53.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:52 smithi072 ceph-mon[33912]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:53.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:52 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:20:53.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:52 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:20:53.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:52 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:20:54.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:53 smithi092 ceph-mon[33665]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:54.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:53 smithi045 ceph-mon[30165]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:54.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:53 smithi072 ceph-mon[33912]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:20:55.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:54 smithi092 ceph-mon[33665]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:55.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:54 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/3508652587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"} : dispatch 2024-05-21T12:20:55.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:54 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"} : dispatch 2024-05-21T12:20:55.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:54 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"}]': finished 2024-05-21T12:20:55.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:54 smithi092 ceph-mon[33665]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T12:20:55.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:54 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:20:55.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:54 smithi045 ceph-mon[30165]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:55.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:54 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/3508652587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"} : dispatch 2024-05-21T12:20:55.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:54 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"} : dispatch 2024-05-21T12:20:55.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:54 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"}]': finished 2024-05-21T12:20:55.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:54 smithi045 ceph-mon[30165]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T12:20:55.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:54 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:20:55.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:54 smithi072 ceph-mon[33912]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:55.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:54 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/3508652587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"} : dispatch 2024-05-21T12:20:55.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:54 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"} : dispatch 2024-05-21T12:20:55.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:54 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf145b0d-6843-4c9f-b33c-9c9129b2c8aa"}]': finished 2024-05-21T12:20:55.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:54 smithi072 ceph-mon[33912]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T12:20:55.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:54 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:20:56.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:55 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/1502673140' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:56.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:55 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/1502673140' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:56.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:55 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/1502673140' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:20:57.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:56 smithi092 ceph-mon[33665]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:57.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:56 smithi045 ceph-mon[30165]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:57.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:56 smithi072 ceph-mon[33912]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:58.897 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:20:58 smithi092 ceph-mon[33665]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:59.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:20:58 smithi045 ceph-mon[30165]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:20:59.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:20:58 smithi072 ceph-mon[33912]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:00.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:00 smithi072 ceph-mon[33912]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:00.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:00 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-05-21T12:21:00.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:00 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:01.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:00 smithi092 ceph-mon[33665]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:01.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:00 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-05-21T12:21:01.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:00 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:01.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:00 smithi045 ceph-mon[30165]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:00 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-05-21T12:21:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:00 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:02.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:01 smithi092 ceph-mon[33665]: Deploying daemon osd.5 on smithi092 2024-05-21T12:21:02.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:01 smithi045 ceph-mon[30165]: Deploying daemon osd.5 on smithi092 2024-05-21T12:21:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:01 smithi072 ceph-mon[33912]: Deploying daemon osd.5 on smithi092 2024-05-21T12:21:03.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:02 smithi092 ceph-mon[33665]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:03.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:02 smithi045 ceph-mon[30165]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:03.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:02 smithi072 ceph-mon[33912]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:05.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:04 smithi092 ceph-mon[33665]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:05.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:04 smithi045 ceph-mon[30165]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:05.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:04 smithi072 ceph-mon[33912]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:07.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:06 smithi092 ceph-mon[33665]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:07.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:06 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:07.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:06 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:07.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:06 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:07.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:06 smithi045 ceph-mon[30165]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:07.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:06 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:07.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:06 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:07.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:06 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:07.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:06 smithi072 ceph-mon[33912]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:07.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:06 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:07.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:06 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:07.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:06 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:08 smithi092 ceph-mon[33665]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:09.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:09.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:08 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:08 smithi045 ceph-mon[30165]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:09.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:09.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:09.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:08 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:08 smithi072 ceph-mon[33912]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:09.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:09.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:09.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:08 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.357 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 5 on host 'smithi092' 2024-05-21T12:21:09.952 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:09 smithi092 ceph-mon[33665]: from='osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-05-21T12:21:09.952 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:09 smithi092 ceph-mon[33665]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-05-21T12:21:09.952 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:09 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.952 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:09 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:09.952 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:09 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:09.978 DEBUG:teuthology.orchestra.run.smithi092:osd.5> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.5.service 2024-05-21T12:21:09.980 INFO:tasks.cephadm:Deploying osd.6 on smithi092 with /dev/vg_nvme/lv_3... 2024-05-21T12:21:09.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T12:21:10.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:09 smithi045 ceph-mon[30165]: from='osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-05-21T12:21:10.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:09 smithi045 ceph-mon[30165]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-05-21T12:21:10.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:09 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:10.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:09 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:10.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:09 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:10.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:09 smithi072 ceph-mon[33912]: from='osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-05-21T12:21:10.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:09 smithi072 ceph-mon[33912]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-05-21T12:21:10.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:09 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:10.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:09 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:10.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:09 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:11.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:10 smithi045 ceph-mon[30165]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:11.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:10 smithi045 ceph-mon[30165]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T12:21:11.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:10 smithi045 ceph-mon[30165]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T12:21:11.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:10 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:11.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:10 smithi045 ceph-mon[30165]: from='osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:11.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:10 smithi045 ceph-mon[30165]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:11.083 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:10 smithi092 ceph-mon[33665]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:11.083 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:10 smithi092 ceph-mon[33665]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T12:21:11.083 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:10 smithi092 ceph-mon[33665]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T12:21:11.083 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:10 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:11.083 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:10 smithi092 ceph-mon[33665]: from='osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:11.083 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:10 smithi092 ceph-mon[33665]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:11.084 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:21:10 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:21:10.762+0000 7f8ba0f83700 -1 osd.5 0 waiting for initial osdmap 2024-05-21T12:21:11.084 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:21:10 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:21:10.766+0000 7f8b9cd76700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T12:21:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:21:11.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:10 smithi072 ceph-mon[33912]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:11.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:10 smithi072 ceph-mon[33912]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T12:21:11.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:10 smithi072 ceph-mon[33912]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T12:21:11.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:10 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:11.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:10 smithi072 ceph-mon[33912]: from='osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:11.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:10 smithi072 ceph-mon[33912]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:11.964 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:11 smithi092 ceph-mon[33665]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:21:11.964 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:11 smithi092 ceph-mon[33665]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T12:21:11.965 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:11.965 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:12.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:11 smithi045 ceph-mon[30165]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:21:12.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:11 smithi045 ceph-mon[30165]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T12:21:12.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:12.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:12.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:11 smithi072 ceph-mon[33912]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:21:12.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:11 smithi072 ceph-mon[33912]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T12:21:12.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:12.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:13.022 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:12 smithi092 ceph-mon[33665]: purged_snaps scrub starts 2024-05-21T12:21:13.022 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:12 smithi092 ceph-mon[33665]: purged_snaps scrub ok 2024-05-21T12:21:13.022 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:12 smithi092 ceph-mon[33665]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:13.022 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:12 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:13.022 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:12 smithi092 ceph-mon[33665]: osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946] boot 2024-05-21T12:21:13.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:12 smithi092 ceph-mon[33665]: osdmap e37: 6 total, 6 up, 6 in 2024-05-21T12:21:13.023 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:12 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:13.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:12 smithi045 ceph-mon[30165]: purged_snaps scrub starts 2024-05-21T12:21:13.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:12 smithi045 ceph-mon[30165]: purged_snaps scrub ok 2024-05-21T12:21:13.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:12 smithi045 ceph-mon[30165]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:13.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:12 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:13.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:12 smithi045 ceph-mon[30165]: osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946] boot 2024-05-21T12:21:13.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:12 smithi045 ceph-mon[30165]: osdmap e37: 6 total, 6 up, 6 in 2024-05-21T12:21:13.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:12 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:12 smithi072 ceph-mon[33912]: purged_snaps scrub starts 2024-05-21T12:21:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:12 smithi072 ceph-mon[33912]: purged_snaps scrub ok 2024-05-21T12:21:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:12 smithi072 ceph-mon[33912]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-21T12:21:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:12 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:12 smithi072 ceph-mon[33912]: osd.5 [v2:172.21.15.92:6800/935256946,v1:172.21.15.92:6801/935256946] boot 2024-05-21T12:21:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:12 smithi072 ceph-mon[33912]: osdmap e37: 6 total, 6 up, 6 in 2024-05-21T12:21:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:12 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-05-21T12:21:13.844 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T12:21:13.844 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T12:21:13.844 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-21T12:21:13.845 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-21T12:21:13.845 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114896 s, 913 MB/s 2024-05-21T12:21:13.845 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-21T12:21:14.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:13 smithi045 ceph-mon[30165]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T12:21:14.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:13 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:13 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:13 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:14.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:13 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:13 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:14.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:13 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:14.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:13 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:13 smithi072 ceph-mon[33912]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T12:21:14.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:13 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:13 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:13 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:14.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:13 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:13 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:14.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:13 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:14.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:13 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.169 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:13 smithi092 ceph-mon[33665]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T12:21:14.169 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:13 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.170 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:13 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.170 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:13 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:14.170 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:13 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.170 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:13 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:14.170 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:13 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:14.170 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:13 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:14.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-05-21T12:21:14.689 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:21:15.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:14 smithi092 ceph-mon[33665]: Detected new or changed devices on smithi092 2024-05-21T12:21:15.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:14 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi092 to 21184M 2024-05-21T12:21:15.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:14 smithi092 ceph-mon[33665]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:15.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:14 smithi092 ceph-mon[33665]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T12:21:15.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:14 smithi045 ceph-mon[30165]: Detected new or changed devices on smithi092 2024-05-21T12:21:15.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:14 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi092 to 21184M 2024-05-21T12:21:15.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:14 smithi045 ceph-mon[30165]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:15.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:14 smithi045 ceph-mon[30165]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T12:21:15.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:14 smithi072 ceph-mon[33912]: Detected new or changed devices on smithi092 2024-05-21T12:21:15.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:14 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi092 to 21184M 2024-05-21T12:21:15.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:14 smithi072 ceph-mon[33912]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:15.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:14 smithi072 ceph-mon[33912]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T12:21:17.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:16 smithi092 ceph-mon[33665]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-05-21T12:21:17.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:16 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:21:17.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:16 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:21:17.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:16 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:17.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:16 smithi045 ceph-mon[30165]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-05-21T12:21:17.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:16 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:21:17.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:16 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:21:17.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:16 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:17.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:16 smithi072 ceph-mon[33912]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-05-21T12:21:17.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:16 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:21:17.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:16 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:21:17.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:16 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:18.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:17 smithi045 ceph-mon[30165]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:21:18.067 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:17 smithi072 ceph-mon[33912]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:21:18.139 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:17 smithi092 ceph-mon[33665]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:21:19.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:18 smithi045 ceph-mon[30165]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T12:21:19.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:18 smithi072 ceph-mon[33912]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T12:21:19.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:18 smithi092 ceph-mon[33665]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T12:21:20.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:19 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/3476580066' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"} : dispatch 2024-05-21T12:21:20.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:19 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"} : dispatch 2024-05-21T12:21:20.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:19 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"}]': finished 2024-05-21T12:21:20.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:19 smithi045 ceph-mon[30165]: osdmap e40: 7 total, 6 up, 7 in 2024-05-21T12:21:20.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:19 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:20.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:19 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/3476580066' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"} : dispatch 2024-05-21T12:21:20.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:19 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"} : dispatch 2024-05-21T12:21:20.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:19 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"}]': finished 2024-05-21T12:21:20.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:19 smithi072 ceph-mon[33912]: osdmap e40: 7 total, 6 up, 7 in 2024-05-21T12:21:20.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:19 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:20.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:19 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/3476580066' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"} : dispatch 2024-05-21T12:21:20.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:19 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"} : dispatch 2024-05-21T12:21:20.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:19 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae8e8621-eef3-472a-b9e9-cb14a6a8abe3"}]': finished 2024-05-21T12:21:20.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:19 smithi092 ceph-mon[33665]: osdmap e40: 7 total, 6 up, 7 in 2024-05-21T12:21:20.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:19 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:21.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:20 smithi045 ceph-mon[30165]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-21T12:21:21.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:20 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4098258345' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:21:21.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:20 smithi072 ceph-mon[33912]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-21T12:21:21.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:20 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4098258345' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:21:21.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:20 smithi092 ceph-mon[33665]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-21T12:21:21.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:20 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4098258345' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:21:23.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:22 smithi045 ceph-mon[30165]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T12:21:23.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:22 smithi072 ceph-mon[33912]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T12:21:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:22 smithi092 ceph-mon[33665]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T12:21:25.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:24 smithi045 ceph-mon[30165]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-21T12:21:25.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:24 smithi072 ceph-mon[33912]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-21T12:21:25.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:24 smithi092 ceph-mon[33665]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-21T12:21:26.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:25 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-05-21T12:21:26.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:25 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:26.110 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:25 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-05-21T12:21:26.110 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:25 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:26.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:25 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-05-21T12:21:26.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:25 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:27.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:26 smithi045 ceph-mon[30165]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:27.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:26 smithi045 ceph-mon[30165]: Deploying daemon osd.6 on smithi092 2024-05-21T12:21:27.076 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:26 smithi092 ceph-mon[33665]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:27.076 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:26 smithi092 ceph-mon[33665]: Deploying daemon osd.6 on smithi092 2024-05-21T12:21:27.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:26 smithi072 ceph-mon[33912]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:27.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:26 smithi072 ceph-mon[33912]: Deploying daemon osd.6 on smithi092 2024-05-21T12:21:29.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:28 smithi045 ceph-mon[30165]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:29.118 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:28 smithi092 ceph-mon[33665]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:29.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:28 smithi072 ceph-mon[33912]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:30.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:29 smithi072 ceph-mon[33912]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:30.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:29 smithi092 ceph-mon[33665]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:30.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:29 smithi045 ceph-mon[30165]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:31.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:31 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:31.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:31 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:31.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:31 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:32.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:31 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:32.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:31 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:32.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:31 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:32.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:31 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:32.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:31 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:32.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:31 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:32 smithi072 ceph-mon[33912]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:32.928 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:32 smithi092 ceph-mon[33665]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:33.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:32 smithi045 ceph-mon[30165]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:33.457 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 6 on host 'smithi092' 2024-05-21T12:21:34.424 DEBUG:teuthology.orchestra.run.smithi092:osd.6> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.6.service 2024-05-21T12:21:34.426 INFO:tasks.cephadm:Deploying osd.7 on smithi092 with /dev/vg_nvme/lv_2... 2024-05-21T12:21:34.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T12:21:34.718 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:21:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:34 smithi092 ceph-mon[33665]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:34 smithi092 ceph-mon[33665]: from='osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-05-21T12:21:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:34 smithi092 ceph-mon[33665]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-05-21T12:21:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:34 smithi045 ceph-mon[30165]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:34.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:34 smithi045 ceph-mon[30165]: from='osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-05-21T12:21:34.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:34 smithi045 ceph-mon[30165]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-05-21T12:21:34.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:34 smithi072 ceph-mon[33912]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:34 smithi072 ceph-mon[33912]: from='osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-05-21T12:21:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:34 smithi072 ceph-mon[33912]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-05-21T12:21:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:35.850 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:35 smithi092 ceph-mon[33665]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T12:21:35.850 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:35 smithi092 ceph-mon[33665]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T12:21:35.850 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:35.850 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:35 smithi092 ceph-mon[33665]: from='osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:35.850 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:35 smithi092 ceph-mon[33665]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:35.850 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:21:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:21:35.646+0000 7f1113e0f700 -1 osd.6 0 waiting for initial osdmap 2024-05-21T12:21:35.851 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:21:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:21:35.652+0000 7f110fc02700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T12:21:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:35 smithi072 ceph-mon[33912]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T12:21:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:35 smithi072 ceph-mon[33912]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T12:21:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:35 smithi072 ceph-mon[33912]: from='osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:35 smithi072 ceph-mon[33912]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:36.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:35 smithi045 ceph-mon[30165]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T12:21:36.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:35 smithi045 ceph-mon[30165]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T12:21:36.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:36.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:35 smithi045 ceph-mon[30165]: from='osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:36.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:35 smithi045 ceph-mon[30165]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:21:36.851 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:36 smithi092 ceph-mon[33665]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:36.851 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:36 smithi092 ceph-mon[33665]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:21:36.851 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:36 smithi092 ceph-mon[33665]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T12:21:36.852 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:36.852 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:36.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:36 smithi072 ceph-mon[33912]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:36.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:36 smithi072 ceph-mon[33912]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:21:36.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:36 smithi072 ceph-mon[33912]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T12:21:36.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:36.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:37.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:36 smithi045 ceph-mon[30165]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:37.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:36 smithi045 ceph-mon[30165]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:21:37.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:36 smithi045 ceph-mon[30165]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T12:21:37.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:37.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:37.601 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T12:21:37.601 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T12:21:37.601 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-21T12:21:37.601 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-21T12:21:37.601 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012434 s, 843 MB/s 2024-05-21T12:21:37.602 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-21T12:21:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:37 smithi072 ceph-mon[33912]: purged_snaps scrub starts 2024-05-21T12:21:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:37 smithi072 ceph-mon[33912]: purged_snaps scrub ok 2024-05-21T12:21:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:37 smithi072 ceph-mon[33912]: osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745] boot 2024-05-21T12:21:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:37 smithi072 ceph-mon[33912]: osdmap e43: 7 total, 7 up, 7 in 2024-05-21T12:21:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:37.948 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:37 smithi092 ceph-mon[33665]: purged_snaps scrub starts 2024-05-21T12:21:37.948 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:37 smithi092 ceph-mon[33665]: purged_snaps scrub ok 2024-05-21T12:21:37.948 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:37 smithi092 ceph-mon[33665]: osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745] boot 2024-05-21T12:21:37.949 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:37 smithi092 ceph-mon[33665]: osdmap e43: 7 total, 7 up, 7 in 2024-05-21T12:21:37.949 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:38.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:37 smithi045 ceph-mon[30165]: purged_snaps scrub starts 2024-05-21T12:21:38.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:37 smithi045 ceph-mon[30165]: purged_snaps scrub ok 2024-05-21T12:21:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:37 smithi045 ceph-mon[30165]: osd.6 [v2:172.21.15.92:6808/1290651745,v1:172.21.15.92:6809/1290651745] boot 2024-05-21T12:21:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:37 smithi045 ceph-mon[30165]: osdmap e43: 7 total, 7 up, 7 in 2024-05-21T12:21:38.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-05-21T12:21:38.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-05-21T12:21:38.855 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:21:38.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:38 smithi072 ceph-mon[33912]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:38.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:38 smithi072 ceph-mon[33912]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T12:21:38.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:38 smithi092 ceph-mon[33665]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:38.970 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:38 smithi092 ceph-mon[33665]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T12:21:39.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:38 smithi045 ceph-mon[30165]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-21T12:21:39.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:38 smithi045 ceph-mon[30165]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: Detected new or changed devices on smithi092 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi092 to 10592M 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.117 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:40.118 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:40.118 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:40.118 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.118 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:39 smithi092 ceph-mon[33665]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: Detected new or changed devices on smithi092 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi092 to 10592M 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:40.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:40.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:40.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:39 smithi072 ceph-mon[33912]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: Detected new or changed devices on smithi092 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi092 to 10592M 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:40.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:40.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:40.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:40.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:39 smithi045 ceph-mon[30165]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:41.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:41 smithi072 ceph-mon[33912]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:21:41.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:21:41.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:21:41.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:41 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:41.442 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:41 smithi092 ceph-mon[33665]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:21:41.442 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:21:41.442 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:21:41.442 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:41 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:41.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:41 smithi045 ceph-mon[30165]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:21:41.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-05-21T12:21:41.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-05-21T12:21:41.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:41 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:42.194 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:42 smithi092 ceph-mon[33665]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:42.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:42 smithi072 ceph-mon[33912]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:42.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:42 smithi045 ceph-mon[30165]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:43.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:43 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/3537646878' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"} : dispatch 2024-05-21T12:21:43.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:43 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"} : dispatch 2024-05-21T12:21:43.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:43 smithi072 ceph-mon[33912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"}]': finished 2024-05-21T12:21:43.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:43 smithi072 ceph-mon[33912]: osdmap e45: 8 total, 7 up, 8 in 2024-05-21T12:21:43.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:43 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:21:43.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:43 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/3537646878' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"} : dispatch 2024-05-21T12:21:43.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:43 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"} : dispatch 2024-05-21T12:21:43.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:43 smithi092 ceph-mon[33665]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"}]': finished 2024-05-21T12:21:43.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:43 smithi092 ceph-mon[33665]: osdmap e45: 8 total, 7 up, 8 in 2024-05-21T12:21:43.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:43 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:21:43.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:43 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/3537646878' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"} : dispatch 2024-05-21T12:21:43.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:43 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"} : dispatch 2024-05-21T12:21:43.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:43 smithi045 ceph-mon[30165]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ede1629e-ba4e-4748-a152-384de3982b67"}]': finished 2024-05-21T12:21:43.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:43 smithi045 ceph-mon[30165]: osdmap e45: 8 total, 7 up, 8 in 2024-05-21T12:21:43.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:21:44.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:44 smithi072 ceph-mon[33912]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:44.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:44 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/3832530543' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:21:44.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:44 smithi092 ceph-mon[33665]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:44.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:44 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/3832530543' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:21:44.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:44 smithi045 ceph-mon[30165]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:44.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:44 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/3832530543' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-05-21T12:21:46.473 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:46 smithi092 ceph-mon[33665]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:46.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:46 smithi045 ceph-mon[30165]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:46.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:46 smithi072 ceph-mon[33912]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:48.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:48 smithi092 ceph-mon[33665]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:48.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:48 smithi045 ceph-mon[30165]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:48.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:48 smithi072 ceph-mon[33912]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:49.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-05-21T12:21:49.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:49.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-05-21T12:21:49.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-05-21T12:21:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:50.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:50 smithi092 ceph-mon[33665]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:50 smithi092 ceph-mon[33665]: Deploying daemon osd.7 on smithi092 2024-05-21T12:21:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:50 smithi045 ceph-mon[30165]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:50 smithi045 ceph-mon[30165]: Deploying daemon osd.7 on smithi092 2024-05-21T12:21:50.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:50 smithi072 ceph-mon[33912]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:50.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:50 smithi072 ceph-mon[33912]: Deploying daemon osd.7 on smithi092 2024-05-21T12:21:52.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:52 smithi045 ceph-mon[30165]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:52.649 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:52 smithi092 ceph-mon[33665]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:52.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:52 smithi072 ceph-mon[33912]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:54.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:54 smithi092 ceph-mon[33665]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:54 smithi045 ceph-mon[30165]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:54.670 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:54 smithi072 ceph-mon[33912]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:56.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:56.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:56.006 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:56.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:56.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:56.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:56.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:56.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:56.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:57.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:56 smithi092 ceph-mon[33665]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:57.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:56 smithi045 ceph-mon[30165]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:57.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:56 smithi072 ceph-mon[33912]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:58.366 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 7 on host 'smithi092' 2024-05-21T12:21:58.669 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:58.669 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:58 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-05-21T12:21:58.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-05-21T12:21:58.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:58 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:21:58.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:58 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:21:58.992 DEBUG:teuthology.orchestra.run.smithi092:osd.7> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.7.service 2024-05-21T12:21:58.994 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-05-21T12:21:58.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-21T12:21:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:00.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:59 smithi092 ceph-mon[33665]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T12:22:00.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:59 smithi092 ceph-mon[33665]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T12:22:00.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:00.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:59 smithi092 ceph-mon[33665]: from='osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:22:00.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:21:59 smithi092 ceph-mon[33665]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:22:00.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:59 smithi045 ceph-mon[30165]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T12:22:00.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:59 smithi045 ceph-mon[30165]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T12:22:00.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:00.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:59 smithi045 ceph-mon[30165]: from='osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:22:00.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:21:59 smithi045 ceph-mon[30165]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:22:00.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:59 smithi072 ceph-mon[33912]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T12:22:00.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:59 smithi072 ceph-mon[33912]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T12:22:00.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:00.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:59 smithi072 ceph-mon[33912]: from='osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:22:00.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:21:59 smithi072 ceph-mon[33912]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]} : dispatch 2024-05-21T12:22:00.421 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:22:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:22:00.199+0000 7fb053742700 -1 osd.7 0 waiting for initial osdmap 2024-05-21T12:22:00.421 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:22:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:22:00.205+0000 7fb04b52d700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T12:22:01.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:00 smithi092 ceph-mon[33665]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:22:01.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:00 smithi092 ceph-mon[33665]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:22:01.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:00 smithi092 ceph-mon[33665]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T12:22:01.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:00 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:01.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:00 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:01.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:00 smithi092 ceph-mon[33665]: from='osd.7 ' entity='osd.7' 2024-05-21T12:22:01.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:00 smithi045 ceph-mon[30165]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:22:01.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:00 smithi045 ceph-mon[30165]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:22:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:00 smithi045 ceph-mon[30165]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T12:22:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:00 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:00 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:01.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:00 smithi045 ceph-mon[30165]: from='osd.7 ' entity='osd.7' 2024-05-21T12:22:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:22:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:00 smithi072 ceph-mon[33912]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:22:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:00 smithi072 ceph-mon[33912]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-21T12:22:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:00 smithi072 ceph-mon[33912]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T12:22:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:00 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:00 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:00 smithi072 ceph-mon[33912]: from='osd.7 ' entity='osd.7' 2024-05-21T12:22:01.842 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":47,"num_osds":8,"num_up_osds":7,"osd_up_since":1716294096,"num_in_osds":8,"osd_in_since":1716294103,"num_remapped_pgs":0} 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: purged_snaps scrub starts 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: purged_snaps scrub ok 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/4124988563' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339] boot 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: osdmap e48: 8 total, 8 up, 8 in 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:22:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:22:02.004 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: purged_snaps scrub starts 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: purged_snaps scrub ok 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/4124988563' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339] boot 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: osdmap e48: 8 total, 8 up, 8 in 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:22:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:22:02.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: purged_snaps scrub starts 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: purged_snaps scrub ok 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/4124988563' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: osd.7 [v2:172.21.15.92:6816/2555974339,v1:172.21.15.92:6817/2555974339] boot 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: osdmap e48: 8 total, 8 up, 8 in 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-05-21T12:22:02.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:22:02.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:22:02.167 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:22:02.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-21T12:22:03.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:02 smithi045 ceph-mon[30165]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:22:03.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:02 smithi045 ceph-mon[30165]: Detected new or changed devices on smithi092 2024-05-21T12:22:03.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:02 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi092 to 7061M 2024-05-21T12:22:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:03.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:02 smithi072 ceph-mon[33912]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:22:03.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:02 smithi072 ceph-mon[33912]: Detected new or changed devices on smithi092 2024-05-21T12:22:03.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:02 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi092 to 7061M 2024-05-21T12:22:03.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:02 smithi092 ceph-mon[33665]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-05-21T12:22:03.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:02 smithi092 ceph-mon[33665]: Detected new or changed devices on smithi092 2024-05-21T12:22:03.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:02 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi092 to 7061M 2024-05-21T12:22:04.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:03 smithi045 ceph-mon[30165]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T12:22:04.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:03 smithi072 ceph-mon[33912]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T12:22:04.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:03 smithi092 ceph-mon[33665]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T12:22:05.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:04 smithi045 ceph-mon[30165]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:05.126 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:22:05.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:04 smithi072 ceph-mon[33912]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:05.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:04 smithi092 ceph-mon[33665]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1716294121,"num_in_osds":8,"osd_in_since":1716294103,"num_remapped_pgs":0} 2024-05-21T12:22:05.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T12:22:06.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:05 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/4262194473' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-05-21T12:22:06.004 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:06.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:05 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/4262194473' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-05-21T12:22:06.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:05 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/4262194473' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-05-21T12:22:07.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:06 smithi045 ceph-mon[30165]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:07.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:06 smithi072 ceph-mon[33912]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:07.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:06 smithi092 ceph-mon[33665]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:07.879 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:22:07.879 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"fsid":"c7e3752a-176b-11ef-bc9a-c7b262605968","created":"2024-05-21T12:16:32.935414+0000","modified":"2024-05-21T12:22:02.752082+0000","last_up_change":"2024-05-21T12:22:01.202094+0000","last_in_change":"2024-05-21T12:21:43.047436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T12:19:59.281802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a991dfbc-70b8-4c8d-8913-19b971ca07d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6803","nonce":1361765727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6805","nonce":1361765727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6809","nonce":1361765727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6807","nonce":1361765727}]},"public_addr":"172.21.15.45:6803/1361765727","cluster_addr":"172.21.15.45:6805/1361765727","heartbeat_back_addr":"172.21.15.45:6809/1361765727","heartbeat_front_addr":"172.21.15.45:6807/1361765727","state":["exists","up"]},{"osd":1,"uuid":"9ae507f9-6943-42d8-b26e-af49f064dc74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6811","nonce":2489398365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6813","nonce":2489398365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6817","nonce":2489398365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6815","nonce":2489398365}]},"public_addr":"172.21.15.45:6811/2489398365","cluster_addr":"172.21.15.45:6813/2489398365","heartbeat_back_addr":"172.21.15.45:6817/2489398365","heartbeat_front_addr":"172.21.15.45:6815/2489398365","state":["exists","up"]},{"osd":2,"uuid":"1a520290-21ec-4b31-9e77-9d71cb659de6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6801","nonce":630200895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6803","nonce":630200895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6807","nonce":630200895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6805","nonce":630200895}]},"public_addr":"172.21.15.72:6801/630200895","cluster_addr":"172.21.15.72:6803/630200895","heartbeat_back_addr":"172.21.15.72:6807/630200895","heartbeat_front_addr":"172.21.15.72:6805/630200895","state":["exists","up"]},{"osd":3,"uuid":"5006e39c-d6e6-44a6-8e2f-0501b36aa3b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6809","nonce":3263314753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6811","nonce":3263314753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6815","nonce":3263314753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6813","nonce":3263314753}]},"public_addr":"172.21.15.72:6809/3263314753","cluster_addr":"172.21.15.72:6811/3263314753","heartbeat_back_addr":"172.21.15.72:6815/3263314753","heartbeat_front_addr":"172.21.15.72:6813/3263314753","state":["exists","up"]},{"osd":4,"uuid":"188f0178-16be-41fa-8f84-d75443654c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6817","nonce":930805302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6819","nonce":930805302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6823","nonce":930805302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6821","nonce":930805302}]},"public_addr":"172.21.15.72:6817/930805302","cluster_addr":"172.21.15.72:6819/930805302","heartbeat_back_addr":"172.21.15.72:6823/930805302","heartbeat_front_addr":"172.21.15.72:6821/930805302","state":["exists","up"]},{"osd":5,"uuid":"cf145b0d-6843-4c9f-b33c-9c9129b2c8aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6801","nonce":935256946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6803","nonce":935256946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6807","nonce":935256946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6805","nonce":935256946}]},"public_addr":"172.21.15.92:6801/935256946","cluster_addr":"172.21.15.92:6803/935256946","heartbeat_back_addr":"172.21.15.92:6807/935256946","heartbeat_front_addr":"172.21.15.92:6805/935256946","state":["exists","up"]},{"osd":6,"uuid":"ae8e8621-eef3-472a-b9e9-cb14a6a8abe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6809","nonce":1290651745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6811","nonce":1290651745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6815","nonce":1290651745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6813","nonce":1290651745}]},"public_addr":"172.21.15.92:6809/1290651745","cluster_addr":"172.21.15.92:6811/1290651745","heartbeat_back_addr":"172.21.15.92:6815/1290651745","heartbeat_front_addr":"172.21.15.92:6813/1290651745","state":["exists","up"]},{"osd":7,"uuid":"ede1629e-ba4e-4748-a152-384de3982b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6817","nonce":2555974339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6819","nonce":2555974339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6823","nonce":2555974339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6821","nonce":2555974339}]},"public_addr":"172.21.15.92:6817/2555974339","cluster_addr":"172.21.15.92:6819/2555974339","heartbeat_back_addr":"172.21.15.92:6823/2555974339","heartbeat_front_addr":"172.21.15.92:6821/2555974339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:08.499042+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:33.694152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:57.059663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:20:21.871107+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:20:47.116251+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:10.078320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:34.698381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:58.827409+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/143442196":"2024-05-22T12:17:27.180331+0000","172.21.15.45:0/4002202916":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/910361469":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6801/2802277221":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/205396496":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6800/2140295410":"2024-05-22T12:17:27.180331+0000","172.21.15.45:0/3795357730":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/4180956632":"2024-05-22T12:16:59.931557+0000","172.21.15.45:6801/2140295410":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6800/2802277221":"2024-05-22T12:16:59.931557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T12:22:08.520 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T12:19:59.281802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T12:22:08.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-21T12:22:08.769 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:09.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:08 smithi045 ceph-mon[30165]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:09.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:08 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3117822978' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:09.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:08 smithi072 ceph-mon[33912]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:09.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:08 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3117822978' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:09.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:08 smithi092 ceph-mon[33665]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:09.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:08 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3117822978' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:10.461 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-05-21T12:22:11.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:10 smithi045 ceph-mon[30165]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:11.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:10 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2026633566' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-05-21T12:22:11.047 INFO:tasks.cephadm:Setting up client nodes... 2024-05-21T12:22:11.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-21T12:22:11.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:10 smithi072 ceph-mon[33912]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:11.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:10 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2026633566' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-05-21T12:22:11.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:10 smithi092 ceph-mon[33665]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:11.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:10 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2026633566' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-05-21T12:22:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:13.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:12 smithi045 ceph-mon[30165]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:13.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:12 smithi072 ceph-mon[33912]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:13.225 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-21T12:22:13.225 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQD1kUxmdi1TDRAAIlXNQPVJYKoQS6Andc+tpg== 2024-05-21T12:22:13.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:12 smithi092 ceph-mon[33665]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:13.941 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:22:13.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-21T12:22:13.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-21T12:22:13.985 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-21T12:22:14.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:13 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2313063877' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:14.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:13 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2313063877' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:14.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:13 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2313063877' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:14.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:13 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2313063877' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:14.235 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T12:22:14.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:13 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2313063877' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:14.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:13 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2313063877' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:15.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:14 smithi045 ceph-mon[30165]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:15.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:14 smithi072 ceph-mon[33912]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:15.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:14 smithi092 ceph-mon[33665]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:16.266 INFO:teuthology.orchestra.run.smithi072.stdout:[client.1] 2024-05-21T12:22:16.266 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQD4kUxmu/unDxAABNqwHDrQBoPULCFWE6canw== 2024-05-21T12:22:16.824 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:16 smithi072 ceph-mon[33912]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:16.824 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:16 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/217146412' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:16.824 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:16 smithi072 ceph-mon[33912]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:16.824 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:16 smithi072 ceph-mon[33912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:16.852 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:22:16.852 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-21T12:22:16.852 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-21T12:22:16.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-21T12:22:17.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:16 smithi045 ceph-mon[30165]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:17.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:16 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/217146412' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:17.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:16 smithi045 ceph-mon[30165]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:17.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:16 smithi045 ceph-mon[30165]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T12:22:17.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:16 smithi092 ceph-mon[33665]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:17.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:16 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/217146412' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:17.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:16 smithi092 ceph-mon[33665]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:17.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:16 smithi092 ceph-mon[33665]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:18.910 INFO:teuthology.orchestra.run.smithi092.stdout:[client.2] 2024-05-21T12:22:18.910 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQD6kUxmXMYGNhAAP8wHU6vQTvrMH+gksKOUMw== 2024-05-21T12:22:19.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:18 smithi045 ceph-mon[30165]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:19.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:18 smithi072 ceph-mon[33912]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:19.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:18 smithi092 ceph-mon[33665]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:19.479 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:22:19.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-05-21T12:22:19.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-21T12:22:19.517 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.3 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-21T12:22:19.756 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T12:22:20.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:19 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/905171717' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:20.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:19 smithi045 ceph-mon[30165]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:20.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:19 smithi045 ceph-mon[30165]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:20.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:19 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/905171717' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:20.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:19 smithi072 ceph-mon[33912]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:20.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:19 smithi072 ceph-mon[33912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:20.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:19 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/905171717' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:20.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:19 smithi092 ceph-mon[33665]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:20.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:19 smithi092 ceph-mon[33665]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:21.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:20 smithi045 ceph-mon[30165]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:21.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:20 smithi072 ceph-mon[33912]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:21.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:20 smithi092 ceph-mon[33665]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:21.921 INFO:teuthology.orchestra.run.smithi137.stdout:[client.3] 2024-05-21T12:22:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD9kUxm5Ry4NhAArLlfcgzvbRB50fEc19a3NA== 2024-05-21T12:22:22.523 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:22:22.523 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.3.keyring 2024-05-21T12:22:22.523 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-21T12:22:22.581 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-21T12:22:22.582 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-21T12:22:22.582 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-21T12:22:22.582 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-21T12:22:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:22.958 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:22 smithi045 ceph-mon[30165]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:22.958 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:22 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/344966759' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:22.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:22 smithi045 ceph-mon[30165]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:22.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:22 smithi045 ceph-mon[30165]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:23.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:22 smithi072 ceph-mon[33912]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:23.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:22 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/344966759' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:23.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:22 smithi072 ceph-mon[33912]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:23.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:22 smithi072 ceph-mon[33912]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:22 smithi092 ceph-mon[33665]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:22 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/344966759' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:22 smithi092 ceph-mon[33665]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-05-21T12:22:23.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:22 smithi092 ceph-mon[33665]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.3", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T12:22:24.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:22:25.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:24 smithi045 ceph-mon[30165]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:25.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:24 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/4215208987' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-05-21T12:22:25.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:24 smithi072 ceph-mon[33912]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:25.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:24 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/4215208987' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-05-21T12:22:25.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:24 smithi092 ceph-mon[33665]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:25.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:24 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/4215208987' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-05-21T12:22:25.393 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":14,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1323271019},{"type":"v1","addr":"172.21.15.45:6801","nonce":1323271019}]},"active_addr":"172.21.15.45:6801/1323271019","active_change":"2024-05-21T12:17:27.180447+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1941236515}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2468276636}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2541149344}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":171524948}]}]} 2024-05-21T12:22:25.396 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-21T12:22:25.396 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-21T12:22:25.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T12:22:25.644 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:26.960 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:26 smithi045 ceph-mon[30165]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:27.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:26 smithi072 ceph-mon[33912]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:27.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:26 smithi092 ceph-mon[33665]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:22:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"fsid":"c7e3752a-176b-11ef-bc9a-c7b262605968","created":"2024-05-21T12:16:32.935414+0000","modified":"2024-05-21T12:22:02.752082+0000","last_up_change":"2024-05-21T12:22:01.202094+0000","last_in_change":"2024-05-21T12:21:43.047436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T12:19:59.281802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a991dfbc-70b8-4c8d-8913-19b971ca07d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6803","nonce":1361765727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6805","nonce":1361765727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6809","nonce":1361765727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6807","nonce":1361765727}]},"public_addr":"172.21.15.45:6803/1361765727","cluster_addr":"172.21.15.45:6805/1361765727","heartbeat_back_addr":"172.21.15.45:6809/1361765727","heartbeat_front_addr":"172.21.15.45:6807/1361765727","state":["exists","up"]},{"osd":1,"uuid":"9ae507f9-6943-42d8-b26e-af49f064dc74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6811","nonce":2489398365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6813","nonce":2489398365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6817","nonce":2489398365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6815","nonce":2489398365}]},"public_addr":"172.21.15.45:6811/2489398365","cluster_addr":"172.21.15.45:6813/2489398365","heartbeat_back_addr":"172.21.15.45:6817/2489398365","heartbeat_front_addr":"172.21.15.45:6815/2489398365","state":["exists","up"]},{"osd":2,"uuid":"1a520290-21ec-4b31-9e77-9d71cb659de6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6801","nonce":630200895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6803","nonce":630200895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6807","nonce":630200895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6805","nonce":630200895}]},"public_addr":"172.21.15.72:6801/630200895","cluster_addr":"172.21.15.72:6803/630200895","heartbeat_back_addr":"172.21.15.72:6807/630200895","heartbeat_front_addr":"172.21.15.72:6805/630200895","state":["exists","up"]},{"osd":3,"uuid":"5006e39c-d6e6-44a6-8e2f-0501b36aa3b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6809","nonce":3263314753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6811","nonce":3263314753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6815","nonce":3263314753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6813","nonce":3263314753}]},"public_addr":"172.21.15.72:6809/3263314753","cluster_addr":"172.21.15.72:6811/3263314753","heartbeat_back_addr":"172.21.15.72:6815/3263314753","heartbeat_front_addr":"172.21.15.72:6813/3263314753","state":["exists","up"]},{"osd":4,"uuid":"188f0178-16be-41fa-8f84-d75443654c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6817","nonce":930805302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6819","nonce":930805302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6823","nonce":930805302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6821","nonce":930805302}]},"public_addr":"172.21.15.72:6817/930805302","cluster_addr":"172.21.15.72:6819/930805302","heartbeat_back_addr":"172.21.15.72:6823/930805302","heartbeat_front_addr":"172.21.15.72:6821/930805302","state":["exists","up"]},{"osd":5,"uuid":"cf145b0d-6843-4c9f-b33c-9c9129b2c8aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6801","nonce":935256946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6803","nonce":935256946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6807","nonce":935256946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6805","nonce":935256946}]},"public_addr":"172.21.15.92:6801/935256946","cluster_addr":"172.21.15.92:6803/935256946","heartbeat_back_addr":"172.21.15.92:6807/935256946","heartbeat_front_addr":"172.21.15.92:6805/935256946","state":["exists","up"]},{"osd":6,"uuid":"ae8e8621-eef3-472a-b9e9-cb14a6a8abe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6809","nonce":1290651745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6811","nonce":1290651745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6815","nonce":1290651745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6813","nonce":1290651745}]},"public_addr":"172.21.15.92:6809/1290651745","cluster_addr":"172.21.15.92:6811/1290651745","heartbeat_back_addr":"172.21.15.92:6815/1290651745","heartbeat_front_addr":"172.21.15.92:6813/1290651745","state":["exists","up"]},{"osd":7,"uuid":"ede1629e-ba4e-4748-a152-384de3982b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6817","nonce":2555974339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6819","nonce":2555974339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6823","nonce":2555974339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6821","nonce":2555974339}]},"public_addr":"172.21.15.92:6817/2555974339","cluster_addr":"172.21.15.92:6819/2555974339","heartbeat_back_addr":"172.21.15.92:6823/2555974339","heartbeat_front_addr":"172.21.15.92:6821/2555974339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:08.499042+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:33.694152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:57.059663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:20:21.871107+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:20:47.116251+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:10.078320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:34.698381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:58.827409+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/143442196":"2024-05-22T12:17:27.180331+0000","172.21.15.45:0/4002202916":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/910361469":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6801/2802277221":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/205396496":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6800/2140295410":"2024-05-22T12:17:27.180331+0000","172.21.15.45:0/3795357730":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/4180956632":"2024-05-22T12:16:59.931557+0000","172.21.15.45:6801/2140295410":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6800/2802277221":"2024-05-22T12:16:59.931557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T12:22:28.028 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:27 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1067469876' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:28.058 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-21T12:22:28.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T12:22:28.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:27 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1067469876' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:28.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:27 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1067469876' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:29.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:28 smithi045 ceph-mon[30165]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:29.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:28 smithi072 ceph-mon[33912]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:29.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:28 smithi092 ceph-mon[33665]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:22:30.097 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"fsid":"c7e3752a-176b-11ef-bc9a-c7b262605968","created":"2024-05-21T12:16:32.935414+0000","modified":"2024-05-21T12:22:02.752082+0000","last_up_change":"2024-05-21T12:22:01.202094+0000","last_in_change":"2024-05-21T12:21:43.047436+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T12:19:59.281802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a991dfbc-70b8-4c8d-8913-19b971ca07d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6803","nonce":1361765727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6805","nonce":1361765727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6809","nonce":1361765727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1361765727},{"type":"v1","addr":"172.21.15.45:6807","nonce":1361765727}]},"public_addr":"172.21.15.45:6803/1361765727","cluster_addr":"172.21.15.45:6805/1361765727","heartbeat_back_addr":"172.21.15.45:6809/1361765727","heartbeat_front_addr":"172.21.15.45:6807/1361765727","state":["exists","up"]},{"osd":1,"uuid":"9ae507f9-6943-42d8-b26e-af49f064dc74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6811","nonce":2489398365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6813","nonce":2489398365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6817","nonce":2489398365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2489398365},{"type":"v1","addr":"172.21.15.45:6815","nonce":2489398365}]},"public_addr":"172.21.15.45:6811/2489398365","cluster_addr":"172.21.15.45:6813/2489398365","heartbeat_back_addr":"172.21.15.45:6817/2489398365","heartbeat_front_addr":"172.21.15.45:6815/2489398365","state":["exists","up"]},{"osd":2,"uuid":"1a520290-21ec-4b31-9e77-9d71cb659de6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6801","nonce":630200895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6803","nonce":630200895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6807","nonce":630200895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":630200895},{"type":"v1","addr":"172.21.15.72:6805","nonce":630200895}]},"public_addr":"172.21.15.72:6801/630200895","cluster_addr":"172.21.15.72:6803/630200895","heartbeat_back_addr":"172.21.15.72:6807/630200895","heartbeat_front_addr":"172.21.15.72:6805/630200895","state":["exists","up"]},{"osd":3,"uuid":"5006e39c-d6e6-44a6-8e2f-0501b36aa3b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6809","nonce":3263314753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6811","nonce":3263314753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6815","nonce":3263314753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":3263314753},{"type":"v1","addr":"172.21.15.72:6813","nonce":3263314753}]},"public_addr":"172.21.15.72:6809/3263314753","cluster_addr":"172.21.15.72:6811/3263314753","heartbeat_back_addr":"172.21.15.72:6815/3263314753","heartbeat_front_addr":"172.21.15.72:6813/3263314753","state":["exists","up"]},{"osd":4,"uuid":"188f0178-16be-41fa-8f84-d75443654c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6817","nonce":930805302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6819","nonce":930805302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6823","nonce":930805302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":930805302},{"type":"v1","addr":"172.21.15.72:6821","nonce":930805302}]},"public_addr":"172.21.15.72:6817/930805302","cluster_addr":"172.21.15.72:6819/930805302","heartbeat_back_addr":"172.21.15.72:6823/930805302","heartbeat_front_addr":"172.21.15.72:6821/930805302","state":["exists","up"]},{"osd":5,"uuid":"cf145b0d-6843-4c9f-b33c-9c9129b2c8aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6801","nonce":935256946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6803","nonce":935256946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6807","nonce":935256946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":935256946},{"type":"v1","addr":"172.21.15.92:6805","nonce":935256946}]},"public_addr":"172.21.15.92:6801/935256946","cluster_addr":"172.21.15.92:6803/935256946","heartbeat_back_addr":"172.21.15.92:6807/935256946","heartbeat_front_addr":"172.21.15.92:6805/935256946","state":["exists","up"]},{"osd":6,"uuid":"ae8e8621-eef3-472a-b9e9-cb14a6a8abe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6809","nonce":1290651745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6811","nonce":1290651745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6815","nonce":1290651745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1290651745},{"type":"v1","addr":"172.21.15.92:6813","nonce":1290651745}]},"public_addr":"172.21.15.92:6809/1290651745","cluster_addr":"172.21.15.92:6811/1290651745","heartbeat_back_addr":"172.21.15.92:6815/1290651745","heartbeat_front_addr":"172.21.15.92:6813/1290651745","state":["exists","up"]},{"osd":7,"uuid":"ede1629e-ba4e-4748-a152-384de3982b67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6817","nonce":2555974339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6819","nonce":2555974339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6823","nonce":2555974339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2555974339},{"type":"v1","addr":"172.21.15.92:6821","nonce":2555974339}]},"public_addr":"172.21.15.92:6817/2555974339","cluster_addr":"172.21.15.92:6819/2555974339","heartbeat_back_addr":"172.21.15.92:6823/2555974339","heartbeat_front_addr":"172.21.15.92:6821/2555974339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:08.499042+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:33.694152+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:19:57.059663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:20:21.871107+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:20:47.116251+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:10.078320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:34.698381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T12:21:58.827409+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/143442196":"2024-05-22T12:17:27.180331+0000","172.21.15.45:0/4002202916":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/910361469":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6801/2802277221":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/205396496":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6800/2140295410":"2024-05-22T12:17:27.180331+0000","172.21.15.45:0/3795357730":"2024-05-22T12:16:59.931557+0000","172.21.15.45:0/4180956632":"2024-05-22T12:16:59.931557+0000","172.21.15.45:6801/2140295410":"2024-05-22T12:17:27.180331+0000","172.21.15.45:6800/2802277221":"2024-05-22T12:16:59.931557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T12:22:30.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-21T12:22:30.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-21T12:22:30.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-21T12:22:30.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-21T12:22:30.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-21T12:22:30.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-21T12:22:30.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-21T12:22:30.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-21T12:22:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:30 smithi045 ceph-mon[30165]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:30 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/4138229371' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:31.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:30 smithi072 ceph-mon[33912]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:31.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:30 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/4138229371' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:31.197 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:31.197 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:31.198 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:31.221 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:30 smithi092 ceph-mon[33665]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:31.221 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:30 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/4138229371' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-05-21T12:22:33.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:32 smithi045 ceph-mon[30165]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:33.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:32 smithi072 ceph-mon[33912]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:33.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:32 smithi092 ceph-mon[33665]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:35.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:34 smithi045 ceph-mon[30165]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:35.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:34 smithi072 ceph-mon[33912]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:35.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:34 smithi092 ceph-mon[33665]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:36.953 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:36 smithi045 ceph-mon[30165]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:37.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:36 smithi072 ceph-mon[33912]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:37.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:36 smithi092 ceph-mon[33665]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:39.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:38 smithi045 ceph-mon[30165]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:39.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:38 smithi072 ceph-mon[33912]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:39.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:38 smithi092 ceph-mon[33665]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:40.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:40 smithi045 ceph-mon[30165]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:41.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:40 smithi072 ceph-mon[33912]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:41.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:40 smithi092 ceph-mon[33665]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:42.154 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:41 smithi045 ceph-mon[30165]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:42.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:41 smithi072 ceph-mon[33912]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:42.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:41 smithi092 ceph-mon[33665]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:42.639 INFO:teuthology.orchestra.run.smithi045.stdout:34359738413 2024-05-21T12:22:42.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-21T12:22:44.441 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:44 smithi045 ceph-mon[30165]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:44.479 INFO:teuthology.orchestra.run.smithi045.stdout:111669149727 2024-05-21T12:22:44.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-21T12:22:44.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:44 smithi072 ceph-mon[33912]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:44 smithi092 ceph-mon[33665]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:45.666 INFO:teuthology.orchestra.run.smithi045.stdout:137438953498 2024-05-21T12:22:45.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-21T12:22:46.020 INFO:teuthology.orchestra.run.smithi045.stdout:158913789974 2024-05-21T12:22:46.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-21T12:22:46.410 INFO:teuthology.orchestra.run.smithi045.stdout:206158430220 2024-05-21T12:22:46.410 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-21T12:22:46.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:46 smithi045 ceph-mon[30165]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:46.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:46 smithi072 ceph-mon[33912]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:46 smithi092 ceph-mon[33665]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:47.263 INFO:teuthology.orchestra.run.smithi045.stdout:55834574888 2024-05-21T12:22:47.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-21T12:22:47.889 INFO:teuthology.orchestra.run.smithi045.stdout:77309411363 2024-05-21T12:22:47.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-21T12:22:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:48.304 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:48.318 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:48.322 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:48.326 INFO:teuthology.orchestra.run.smithi045.stdout:184683593744 2024-05-21T12:22:48.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-21T12:22:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:48.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:48 smithi045 ceph-mon[30165]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:48.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:48 smithi072 ceph-mon[33912]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:48 smithi092 ceph-mon[33665]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:50.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:50 smithi045 ceph-mon[30165]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:50.554 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:50 smithi092 ceph-mon[33665]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:50.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:50 smithi072 ceph-mon[33912]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:52.381 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:52 smithi045 ceph-mon[30165]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:52.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:52 smithi072 ceph-mon[33912]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:52 smithi092 ceph-mon[33665]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:53.104 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:54 smithi045 ceph-mon[30165]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:54.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:54 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2621431023' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-05-21T12:22:54.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:54 smithi072 ceph-mon[33912]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:54.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:54 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2621431023' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-05-21T12:22:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:54 smithi092 ceph-mon[33665]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:54 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2621431023' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-05-21T12:22:55.137 INFO:teuthology.orchestra.run.smithi045.stdout:111669149729 2024-05-21T12:22:55.241 INFO:teuthology.orchestra.run.smithi045.stdout:206158430222 2024-05-21T12:22:55.363 INFO:teuthology.orchestra.run.smithi045.stdout:137438953501 2024-05-21T12:22:55.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:55 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3306410193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-05-21T12:22:55.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:55 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3306410193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-05-21T12:22:55.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:55 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3306410193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-05-21T12:22:55.817 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430220 got 206158430222 for osd.7 2024-05-21T12:22:55.818 DEBUG:teuthology.parallel:result is None 2024-05-21T12:22:56.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:56 smithi045 ceph-mon[30165]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:56.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:56 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/848728124' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-05-21T12:22:56.526 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149727 got 111669149729 for osd.3 2024-05-21T12:22:56.527 DEBUG:teuthology.parallel:result is None 2024-05-21T12:22:56.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:56 smithi072 ceph-mon[33912]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:56.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:56 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/848728124' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-05-21T12:22:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:56 smithi092 ceph-mon[33665]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:56 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/848728124' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-05-21T12:22:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:22:57.323 INFO:teuthology.orchestra.run.smithi045.stdout:158913789976 2024-05-21T12:22:58.190 INFO:teuthology.orchestra.run.smithi045.stdout:34359738416 2024-05-21T12:22:58.492 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:58 smithi045 ceph-mon[30165]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:58.492 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:58 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3314209193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-05-21T12:22:58.492 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:22:58 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3823413360' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-05-21T12:22:58.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953498 got 137438953501 for osd.4 2024-05-21T12:22:58.519 DEBUG:teuthology.parallel:result is None 2024-05-21T12:22:58.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:58 smithi072 ceph-mon[33912]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:58.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:58 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3314209193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-05-21T12:22:58.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:22:58 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3823413360' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-05-21T12:22:58.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:58 smithi092 ceph-mon[33665]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:22:58.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:58 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3314209193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-05-21T12:22:58.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:22:58 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3823413360' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-05-21T12:22:59.811 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738413 got 34359738416 for osd.0 2024-05-21T12:22:59.811 DEBUG:teuthology.parallel:result is None 2024-05-21T12:23:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:00 smithi045 ceph-mon[30165]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:00.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:00 smithi072 ceph-mon[33912]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:00.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:00 smithi092 ceph-mon[33665]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:02.210 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789974 got 158913789976 for osd.5 2024-05-21T12:23:02.211 DEBUG:teuthology.parallel:result is None 2024-05-21T12:23:02.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:02 smithi045 ceph-mon[30165]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:02.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:02.642 INFO:teuthology.orchestra.run.smithi045.stdout:55834574891 2024-05-21T12:23:02.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:02 smithi072 ceph-mon[33912]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:02.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:02 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:02 smithi092 ceph-mon[33665]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:02 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:02.861 INFO:teuthology.orchestra.run.smithi045.stdout:184683593747 2024-05-21T12:23:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:03 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3991067303' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-05-21T12:23:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:03 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3843383045' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-05-21T12:23:03.526 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593744 got 184683593747 for osd.6 2024-05-21T12:23:03.527 DEBUG:teuthology.parallel:result is None 2024-05-21T12:23:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:03 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3991067303' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-05-21T12:23:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:03.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:03 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3843383045' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-05-21T12:23:03.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:03 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3991067303' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-05-21T12:23:03.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:03.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:03.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:03.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:03 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3843383045' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-05-21T12:23:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:04 smithi045 ceph-mon[30165]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:04.601 INFO:teuthology.orchestra.run.smithi045.stdout:77309411366 2024-05-21T12:23:04.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:04 smithi072 ceph-mon[33912]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:04 smithi092 ceph-mon[33665]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:04.960 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574888 got 55834574891 for osd.1 2024-05-21T12:23:04.961 DEBUG:teuthology.parallel:result is None 2024-05-21T12:23:05.344 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411363 got 77309411366 for osd.2 2024-05-21T12:23:05.344 DEBUG:teuthology.parallel:result is None 2024-05-21T12:23:05.344 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T12:23:05.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T12:23:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:05 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1494386708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-05-21T12:23:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:05.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:05 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1494386708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-05-21T12:23:05.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:05 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1494386708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-05-21T12:23:06.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:06 smithi072 ceph-mon[33912]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:06.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:06 smithi045 ceph-mon[30165]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:06 smithi092 ceph-mon[33665]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:07.449 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:23:07.451 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-21T12:23:08.020 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-05-21T12:23:07.244227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":221568,"kb_used_data":5388,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749739648,"statfs":{"total":767960285184,"available":767733399552,"internally_reserved":0,"allocated":5517312,"data_stored":3288784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002136"},"pg_stats":[{"pgid":"1.0","version":"20'92","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-21T12:22:04.058466+0000","last_change":"2024-05-21T12:21:13.782098+0000","last_active":"2024-05-21T12:22:04.058466+0000","last_peered":"2024-05-21T12:22:04.058466+0000","last_clean":"2024-05-21T12:22:04.058466+0000","last_became_active":"2024-05-21T12:21:13.776240+0000","last_became_peered":"2024-05-21T12:21:13.776240+0000","last_unstale":"2024-05-21T12:22:04.058466+0000","last_undegraded":"2024-05-21T12:22:04.058466+0000","last_fullsized":"2024-05-21T12:22:04.058466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T12:19:59.781605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T12:19:59.781605+0000","last_clean_scrub_stamp":"2024-05-21T12:19:59.781605+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:43:15.950698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717668,"statfs":{"total":95995035648,"available":95966892032,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593748,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717672,"statfs":{"total":95995035648,"available":95966896128,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717676,"statfs":{"total":95995035648,"available":95966900224,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717676,"statfs":{"total":95995035648,"available":95966900224,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411367,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":780078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":780078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717672,"statfs":{"total":95995035648,"available":95966896128,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789978,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":780078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T12:23:08.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T12:23:08.270 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:08.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:08 smithi045 ceph-mon[30165]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:08.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:08 smithi045 ceph-mon[30165]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:23:08.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:08 smithi072 ceph-mon[33912]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:08.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:08 smithi072 ceph-mon[33912]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:23:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:08 smithi092 ceph-mon[33665]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:08 smithi092 ceph-mon[33665]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:23:10.256 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:23:10.258 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-21T12:23:10.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:10 smithi045 ceph-mon[30165]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:10.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:10 smithi072 ceph-mon[33912]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:10 smithi092 ceph-mon[33665]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:10.952 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-05-21T12:23:09.244526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":221568,"kb_used_data":5388,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749739648,"statfs":{"total":767960285184,"available":767733399552,"internally_reserved":0,"allocated":5517312,"data_stored":3288784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002075"},"pg_stats":[{"pgid":"1.0","version":"20'92","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-21T12:22:04.058466+0000","last_change":"2024-05-21T12:21:13.782098+0000","last_active":"2024-05-21T12:22:04.058466+0000","last_peered":"2024-05-21T12:22:04.058466+0000","last_clean":"2024-05-21T12:22:04.058466+0000","last_became_active":"2024-05-21T12:21:13.776240+0000","last_became_peered":"2024-05-21T12:21:13.776240+0000","last_unstale":"2024-05-21T12:22:04.058466+0000","last_undegraded":"2024-05-21T12:22:04.058466+0000","last_fullsized":"2024-05-21T12:22:04.058466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T12:19:59.781605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T12:19:59.781605+0000","last_clean_scrub_stamp":"2024-05-21T12:19:59.781605+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:43:15.950698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717668,"statfs":{"total":95995035648,"available":95966892032,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717672,"statfs":{"total":95995035648,"available":95966896128,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717676,"statfs":{"total":95995035648,"available":95966900224,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717676,"statfs":{"total":95995035648,"available":95966900224,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411368,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":780078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":780078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717672,"statfs":{"total":95995035648,"available":95966896128,"internally_reserved":0,"allocated":466944,"data_stored":189710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789979,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":780078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T12:23:10.953 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T12:23:10.953 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-21T12:23:10.953 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-21T12:23:10.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-21T12:23:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:11.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:11 smithi045 ceph-mon[30165]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:23:11.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:11 smithi072 ceph-mon[33912]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:23:11.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:11 smithi092 ceph-mon[33665]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:23:12.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:12 smithi045 ceph-mon[30165]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:12 smithi072 ceph-mon[33912]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:12 smithi092 ceph-mon[33665]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:13.266 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:23:13.266 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-21T12:23:13.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:13 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/870769010' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-05-21T12:23:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:13 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/870769010' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-05-21T12:23:13.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:13 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/870769010' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-05-21T12:23:13.842 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-21T12:23:13.842 INFO:tasks.cephadm:Setup complete, yielding 2024-05-21T12:23:13.843 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T12:23:13.913 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi045.front.sepia.ceph.com 2024-05-21T12:23:13.913 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-21T12:23:14.159 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:14 smithi045 ceph-mon[30165]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:14.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:14 smithi072 ceph-mon[33912]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:14 smithi092 ceph-mon[33665]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:16.018 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2024-05-21T12:23:16.018 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2024-05-21T12:23:16.018 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2024-05-21T12:23:16.665 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:16 smithi045 ceph-mon[30165]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:16 smithi072 ceph-mon[33912]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:16.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-21T12:23:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:16 smithi092 ceph-mon[33665]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:16.940 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:17.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:17 smithi072 ceph-mon[33912]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:17.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:17 smithi092 ceph-mon[33665]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:17.788 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:17 smithi045 ceph-mon[30165]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:18.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:18 smithi072 ceph-mon[33912]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:18 smithi092 ceph-mon[33665]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:18 smithi045 ceph-mon[30165]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (6m) 3m ago 6m 509M - 19.0.0-3665-gb0c764b6 1238c5f3fda9 87ed08596caf 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (6m) 3m ago 6m 38.8M 2048M 19.0.0-3665-gb0c764b6 1238c5f3fda9 80d88adb7cd3 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi072 running (4m) 2m ago 4m 35.9M 2048M 19.0.0-3665-gb0c764b6 1238c5f3fda9 a8904b856454 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi092 running (4m) 81s ago 4m 36.8M 2048M 19.0.0-3665-gb0c764b6 1238c5f3fda9 9de20c300701 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (4m) 3m ago 4m 25.5M 8546M 19.0.0-3665-gb0c764b6 1238c5f3fda9 c40f57778242 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi045 running (3m) 3m ago 3m 15.0M 8546M 19.0.0-3665-gb0c764b6 1238c5f3fda9 fcded796e72b 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi072 running (3m) 2m ago 3m 35.5M 7061M 19.0.0-3665-gb0c764b6 1238c5f3fda9 ffe9aa9c8813 2024-05-21T12:23:18.831 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi072 running (3m) 2m ago 3m 41.7M 7061M 19.0.0-3665-gb0c764b6 1238c5f3fda9 8144547dcfbd 2024-05-21T12:23:18.832 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi072 running (2m) 2m ago 2m 15.1M 7061M 19.0.0-3665-gb0c764b6 1238c5f3fda9 0ec103327324 2024-05-21T12:23:18.832 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi092 running (2m) 81s ago 2m 36.7M 7061M 19.0.0-3665-gb0c764b6 1238c5f3fda9 53fc0870708f 2024-05-21T12:23:18.832 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi092 running (108s) 81s ago 108s 31.3M 7061M 19.0.0-3665-gb0c764b6 1238c5f3fda9 a15b4b6a2f7a 2024-05-21T12:23:18.832 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi092 running (84s) 81s ago 84s 12.2M 7061M 19.0.0-3665-gb0c764b6 1238c5f3fda9 30c13c67ca7b 2024-05-21T12:23:19.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-21T12:23:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:19 smithi045 ceph-mon[30165]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:19.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:19 smithi072 ceph-mon[33912]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:19.687 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:19.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:19 smithi092 ceph-mon[33665]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:20.653 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:20 smithi092 ceph-mon[33665]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:20.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:20 smithi072 ceph-mon[33912]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:20.760 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:20 smithi045 ceph-mon[30165]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:21.499 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2024-05-21T12:23:21.500 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2024-05-21T12:23:21.500 INFO:teuthology.orchestra.run.smithi045.stdout:smithi072 172.21.15.72 2024-05-21T12:23:21.500 INFO:teuthology.orchestra.run.smithi045.stdout:smithi092 172.21.15.92 2024-05-21T12:23:21.500 INFO:teuthology.orchestra.run.smithi045.stdout:smithi137 172.21.15.137 2024-05-21T12:23:21.500 INFO:teuthology.orchestra.run.smithi045.stdout:4 hosts in cluster 2024-05-21T12:23:22.169 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-21T12:23:22.432 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:22.482 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:22 smithi045 ceph-mon[30165]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:22.482 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:22 smithi045 ceph-mon[30165]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:22.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:22 smithi072 ceph-mon[33912]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:22.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:22 smithi072 ceph-mon[33912]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:22 smithi092 ceph-mon[33665]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:22 smithi092 ceph-mon[33665]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:24.226 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-21T12:23:24.226 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-05-21T12:23:24.226 INFO:teuthology.orchestra.run.smithi045.stdout:smithi072 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V7400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-21T12:23:24.227 INFO:teuthology.orchestra.run.smithi045.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 82s ago Has a FileSystem, LVM detected 2024-05-21T12:23:24.227 INFO:teuthology.orchestra.run.smithi045.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-21T12:23:24.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:24 smithi045 ceph-mon[30165]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:24.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:24 smithi072 ceph-mon[33912]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:24 smithi092 ceph-mon[33665]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:24.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd lspools' 2024-05-21T12:23:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:25.456 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:25 smithi045 ceph-mon[30165]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:25.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:25 smithi072 ceph-mon[33912]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:25.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:25 smithi092 ceph-mon[33665]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:26.607 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:26 smithi045 ceph-mon[30165]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:26 smithi072 ceph-mon[33912]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:26 smithi092 ceph-mon[33665]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:26.849 INFO:teuthology.orchestra.run.smithi045.stdout:1 .mgr 2024-05-21T12:23:27.449 INFO:teuthology.run_tasks:Running task nvmeof... 2024-05-21T12:23:27.552 INFO:tasks.nvmeof:[nvmeof]: deploying nvmeof gateway... 2024-05-21T12:23:27.553 INFO:tasks.nvmeof:Adding ceph.nvmeof.nvmeof.a on smithi045 2024-05-21T12:23:27.553 INFO:tasks.nvmeof:Adding ceph.nvmeof.nvmeof.b on smithi072 2024-05-21T12:23:27.553 INFO:tasks.nvmeof:Adding ceph.nvmeof.nvmeof.c on smithi092 2024-05-21T12:23:27.553 INFO:tasks.nvmeof:[nvmeof]: ceph config set mgr mgr/cephadm/container_image_nvmeof quay.io/ceph/nvmeof:1.2 2024-05-21T12:23:27.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/container_image_nvmeof quay.io/ceph/nvmeof:1.2 2024-05-21T12:23:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:27 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2085188233' entity='client.admin' cmd={"prefix": "osd lspools"} : dispatch 2024-05-21T12:23:27.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:27 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2085188233' entity='client.admin' cmd={"prefix": "osd lspools"} : dispatch 2024-05-21T12:23:27.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:27 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2085188233' entity='client.admin' cmd={"prefix": "osd lspools"} : dispatch 2024-05-21T12:23:27.804 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:28 smithi045 ceph-mon[30165]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:28.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:28 smithi072 ceph-mon[33912]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:28 smithi092 ceph-mon[33665]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:30.228 INFO:tasks.nvmeof:[nvmeof]: ceph osd pool create mypool 2024-05-21T12:23:30.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph osd pool create mypool 2024-05-21T12:23:30.513 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:30.869 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:30 smithi045 ceph-mon[30165]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:30.869 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:30 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3983397096' entity='client.admin' 2024-05-21T12:23:30.869 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:30.869 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:30.869 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:30.869 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:30 smithi072 ceph-mon[33912]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:30 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3983397096' entity='client.admin' 2024-05-21T12:23:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:31.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:30 smithi092 ceph-mon[33665]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:31.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:30 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3983397096' entity='client.admin' 2024-05-21T12:23:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:32.592 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'mypool' created 2024-05-21T12:23:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:32 smithi045 ceph-mon[30165]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:32 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2125483658' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "mypool"} : dispatch 2024-05-21T12:23:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:32 smithi072 ceph-mon[33912]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:32 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2125483658' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "mypool"} : dispatch 2024-05-21T12:23:33.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:32 smithi092 ceph-mon[33665]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:33.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:32 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2125483658' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "mypool"} : dispatch 2024-05-21T12:23:33.221 INFO:tasks.nvmeof:[nvmeof]: rbd pool init mypool 2024-05-21T12:23:33.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd pool init mypool 2024-05-21T12:23:33.472 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:33.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:33 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2125483658' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "mypool"}]': finished 2024-05-21T12:23:33.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:33 smithi045 ceph-mon[30165]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T12:23:33.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:33 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2125483658' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "mypool"}]': finished 2024-05-21T12:23:33.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:33 smithi072 ceph-mon[33912]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T12:23:34.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:33 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2125483658' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "mypool"}]': finished 2024-05-21T12:23:34.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:33 smithi092 ceph-mon[33665]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T12:23:34.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:34 smithi045 ceph-mon[30165]: pgmap v223: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:34.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:34 smithi045 ceph-mon[30165]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T12:23:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:34 smithi072 ceph-mon[33912]: pgmap v223: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:34.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:34 smithi072 ceph-mon[33912]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T12:23:35.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:34 smithi092 ceph-mon[33665]: pgmap v223: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:35.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:34 smithi092 ceph-mon[33665]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T12:23:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:35 smithi072 ceph-mon[33912]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T12:23:35.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:35 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2068466344' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "mypool","app": "rbd"} : dispatch 2024-05-21T12:23:36.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:35 smithi092 ceph-mon[33665]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T12:23:36.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:35 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2068466344' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "mypool","app": "rbd"} : dispatch 2024-05-21T12:23:36.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:35 smithi045 ceph-mon[30165]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T12:23:36.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:35 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2068466344' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "mypool","app": "rbd"} : dispatch 2024-05-21T12:23:36.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:36 smithi072 ceph-mon[33912]: pgmap v226: 33 pgs: 7 creating+peering, 12 active+clean, 14 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:36.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:36 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2068466344' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "mypool","app": "rbd"}]': finished 2024-05-21T12:23:36.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:36 smithi072 ceph-mon[33912]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T12:23:37.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:36 smithi092 ceph-mon[33665]: pgmap v226: 33 pgs: 7 creating+peering, 12 active+clean, 14 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:37.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:36 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2068466344' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "mypool","app": "rbd"}]': finished 2024-05-21T12:23:37.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:36 smithi092 ceph-mon[33665]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T12:23:37.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:36 smithi045 ceph-mon[30165]: pgmap v226: 33 pgs: 7 creating+peering, 12 active+clean, 14 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:37.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:36 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2068466344' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "mypool","app": "rbd"}]': finished 2024-05-21T12:23:37.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:36 smithi045 ceph-mon[30165]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T12:23:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:37 smithi072 ceph-mon[33912]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T12:23:37.926 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:37 smithi045 ceph-mon[30165]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T12:23:38.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:37 smithi092 ceph-mon[33665]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T12:23:38.329 INFO:tasks.nvmeof:[nvmeof]: ceph orch apply nvmeof mypool 2024-05-21T12:23:38.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch apply nvmeof mypool --placement '3;smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c' 2024-05-21T12:23:38.585 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:38.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:38 smithi072 ceph-mon[33912]: pgmap v229: 33 pgs: 7 creating+peering, 26 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:38.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:38 smithi072 ceph-mon[33912]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T12:23:38.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:38 smithi045 ceph-mon[30165]: pgmap v229: 33 pgs: 7 creating+peering, 26 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:38.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:38 smithi045 ceph-mon[30165]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T12:23:39.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:38 smithi092 ceph-mon[33665]: pgmap v229: 33 pgs: 7 creating+peering, 26 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:39.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:38 smithi092 ceph-mon[33665]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T12:23:40.482 INFO:teuthology.orchestra.run.smithi045.stdout:Scheduled nvmeof.mypool update... 2024-05-21T12:23:40.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: pgmap v231: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:40.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:40.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:40.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:40.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:40.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:40.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.c", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:23:40.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.c", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:23:40.917 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:40 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:40.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: pgmap v231: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:40.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:40.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:40.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:40.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:40.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:40.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.c", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:23:40.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.c", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:23:40.925 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:40 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:41.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: pgmap v231: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-05-21T12:23:41.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:41.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:23:41.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:41.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:23:41.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:41.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.c", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:23:41.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.c", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:23:41.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:40 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:41.093 INFO:tasks.nvmeof:[nvmeof]: creating 60 images 2024-05-21T12:23:41.093 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage1 --size 8192 2024-05-21T12:23:41.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage1 --size 8192 2024-05-21T12:23:41.349 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:41.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:41 smithi045 ceph-mon[30165]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch apply nvmeof", "pool": "mypool", "placement": "3;smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:41.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:41 smithi045 ceph-mon[30165]: Saving service nvmeof.mypool spec with placement smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:23:41.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:41 smithi045 ceph-mon[30165]: Deploying daemon nvmeof.nvmeof.c on smithi092 2024-05-21T12:23:41.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:41 smithi072 ceph-mon[33912]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch apply nvmeof", "pool": "mypool", "placement": "3;smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:41.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:41 smithi072 ceph-mon[33912]: Saving service nvmeof.mypool spec with placement smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:23:41.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:41 smithi072 ceph-mon[33912]: Deploying daemon nvmeof.nvmeof.c on smithi092 2024-05-21T12:23:42.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:41 smithi092 ceph-mon[33665]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch apply nvmeof", "pool": "mypool", "placement": "3;smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:23:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:41 smithi092 ceph-mon[33665]: Saving service nvmeof.mypool spec with placement smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:23:42.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:41 smithi092 ceph-mon[33665]: Deploying daemon nvmeof.nvmeof.c on smithi092 2024-05-21T12:23:42.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:42 smithi045 ceph-mon[30165]: pgmap v232: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 341 B/s wr, 0 op/s 2024-05-21T12:23:42.877 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:42 smithi092 ceph-mon[33665]: pgmap v232: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 341 B/s wr, 0 op/s 2024-05-21T12:23:42.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:42 smithi072 ceph-mon[33912]: pgmap v232: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 341 B/s wr, 0 op/s 2024-05-21T12:23:43.578 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage2 --size 8192 2024-05-21T12:23:43.579 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage2 --size 8192 2024-05-21T12:23:43.829 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:44.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:44 smithi045 ceph-mon[30165]: pgmap v233: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 267 B/s wr, 0 op/s 2024-05-21T12:23:44.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:44 smithi072 ceph-mon[33912]: pgmap v233: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 267 B/s wr, 0 op/s 2024-05-21T12:23:45.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:44 smithi092 ceph-mon[33665]: pgmap v233: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 267 B/s wr, 0 op/s 2024-05-21T12:23:45.952 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage3 --size 8192 2024-05-21T12:23:45.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage3 --size 8192 2024-05-21T12:23:46.204 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:46.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:46 smithi072 ceph-mon[33912]: pgmap v234: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 236 B/s rd, 473 B/s wr, 1 op/s 2024-05-21T12:23:47.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:46 smithi092 ceph-mon[33665]: pgmap v234: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 236 B/s rd, 473 B/s wr, 1 op/s 2024-05-21T12:23:47.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:46 smithi045 ceph-mon[30165]: pgmap v234: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 236 B/s rd, 473 B/s wr, 1 op/s 2024-05-21T12:23:48.577 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage4 --size 8192 2024-05-21T12:23:48.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage4 --size 8192 2024-05-21T12:23:48.829 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:48.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:48 smithi072 ceph-mon[33912]: pgmap v235: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 818 B/s wr, 1 op/s 2024-05-21T12:23:48.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:48 smithi072 ceph-mon[33912]: osdmap e56: 8 total, 8 up, 8 in 2024-05-21T12:23:48.951 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:48 smithi045 ceph-mon[30165]: pgmap v235: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 818 B/s wr, 1 op/s 2024-05-21T12:23:48.951 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:48 smithi045 ceph-mon[30165]: osdmap e56: 8 total, 8 up, 8 in 2024-05-21T12:23:49.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:48 smithi092 ceph-mon[33665]: pgmap v235: 33 pgs: 33 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 818 B/s wr, 1 op/s 2024-05-21T12:23:49.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:48 smithi092 ceph-mon[33665]: osdmap e56: 8 total, 8 up, 8 in 2024-05-21T12:23:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:50 smithi072 ceph-mon[33912]: pgmap v237: 33 pgs: 33 active+clean; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 921 B/s wr, 2 op/s 2024-05-21T12:23:50.962 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:50 smithi045 ceph-mon[30165]: pgmap v237: 33 pgs: 33 active+clean; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 921 B/s wr, 2 op/s 2024-05-21T12:23:50.996 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage5 --size 8192 2024-05-21T12:23:50.996 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage5 --size 8192 2024-05-21T12:23:51.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:50 smithi092 ceph-mon[33665]: pgmap v237: 33 pgs: 33 active+clean; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 716 B/s rd, 921 B/s wr, 2 op/s 2024-05-21T12:23:51.253 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:52.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:52 smithi045 ceph-mon[30165]: pgmap v238: 33 pgs: 33 active+clean; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 921 B/s wr, 2 op/s 2024-05-21T12:23:52.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:52 smithi072 ceph-mon[33912]: pgmap v238: 33 pgs: 33 active+clean; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 921 B/s wr, 2 op/s 2024-05-21T12:23:53.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:52 smithi092 ceph-mon[33665]: pgmap v238: 33 pgs: 33 active+clean; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 921 B/s wr, 2 op/s 2024-05-21T12:23:53.609 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage6 --size 8192 2024-05-21T12:23:53.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage6 --size 8192 2024-05-21T12:23:53.860 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:54 smithi092 ceph-mon[33665]: pgmap v239: 33 pgs: 33 active+clean; 579 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:23:54.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:54 smithi045 ceph-mon[30165]: pgmap v239: 33 pgs: 33 active+clean; 579 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:23:54.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:54 smithi072 ceph-mon[33912]: pgmap v239: 33 pgs: 33 active+clean; 579 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:23:56.059 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage7 --size 8192 2024-05-21T12:23:56.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage7 --size 8192 2024-05-21T12:23:56.311 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:23:56.760 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:56 smithi045 ceph-mon[30165]: pgmap v240: 33 pgs: 33 active+clean; 579 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:23:56.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:56 smithi072 ceph-mon[33912]: pgmap v240: 33 pgs: 33 active+clean; 579 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:23:56.922 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:56 smithi092 ceph-mon[33665]: pgmap v240: 33 pgs: 33 active+clean; 579 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:23:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.b", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:23:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.b", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:23:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:57 smithi045 ceph-mon[30165]: Deploying daemon nvmeof.nvmeof.b on smithi072 2024-05-21T12:23:58.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:23:57 smithi045 ceph-mon[30165]: pgmap v241: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-05-21T12:23:58.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.b", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:23:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.b", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:23:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:57 smithi092 ceph-mon[33665]: Deploying daemon nvmeof.nvmeof.b on smithi072 2024-05-21T12:23:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:23:57 smithi092 ceph-mon[33665]: pgmap v241: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-05-21T12:23:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:23:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.b", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:23:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.b", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:23:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:23:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:57 smithi072 ceph-mon[33912]: Deploying daemon nvmeof.nvmeof.b on smithi072 2024-05-21T12:23:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:23:57 smithi072 ceph-mon[33912]: pgmap v241: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-05-21T12:23:58.695 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage8 --size 8192 2024-05-21T12:23:58.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage8 --size 8192 2024-05-21T12:23:58.943 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:00 smithi045 ceph-mon[30165]: pgmap v242: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:24:00.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:00 smithi072 ceph-mon[33912]: pgmap v242: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:24:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:00 smithi092 ceph-mon[33665]: pgmap v242: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-05-21T12:24:00.996 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage9 --size 8192 2024-05-21T12:24:00.996 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage9 --size 8192 2024-05-21T12:24:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:02.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:02 smithi092 ceph-mon[33665]: pgmap v243: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-05-21T12:24:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:02 smithi045 ceph-mon[30165]: pgmap v243: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-05-21T12:24:02.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:02 smithi072 ceph-mon[33912]: pgmap v243: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-05-21T12:24:03.541 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage10 --size 8192 2024-05-21T12:24:03.541 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage10 --size 8192 2024-05-21T12:24:03.791 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:04 smithi045 ceph-mon[30165]: pgmap v244: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:04.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:04 smithi072 ceph-mon[33912]: pgmap v244: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:04 smithi092 ceph-mon[33665]: pgmap v244: 33 pgs: 33 active+clean; 580 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:06.122 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage11 --size 8192 2024-05-21T12:24:06.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage11 --size 8192 2024-05-21T12:24:06.370 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:06.489 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:06 smithi045 ceph-mon[30165]: pgmap v245: 33 pgs: 33 active+clean; 581 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-21T12:24:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:06 smithi072 ceph-mon[33912]: pgmap v245: 33 pgs: 33 active+clean; 581 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-21T12:24:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:06 smithi092 ceph-mon[33665]: pgmap v245: 33 pgs: 33 active+clean; 581 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-21T12:24:08.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:08 smithi045 ceph-mon[30165]: pgmap v246: 33 pgs: 33 active+clean; 582 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:08.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:08 smithi072 ceph-mon[33912]: pgmap v246: 33 pgs: 33 active+clean; 582 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:08.707 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage12 --size 8192 2024-05-21T12:24:08.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage12 --size 8192 2024-05-21T12:24:08.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:08 smithi092 ceph-mon[33665]: pgmap v246: 33 pgs: 33 active+clean; 582 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:10.585 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:10 smithi072 ceph-mon[33912]: pgmap v247: 33 pgs: 33 active+clean; 582 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:10.603 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:10 smithi045 ceph-mon[30165]: pgmap v247: 33 pgs: 33 active+clean; 582 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:10 smithi092 ceph-mon[33665]: pgmap v247: 33 pgs: 33 active+clean; 582 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:11.381 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage13 --size 8192 2024-05-21T12:24:11.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage13 --size 8192 2024-05-21T12:24:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.a", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:24:11.645 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.a", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:24:11.645 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:11 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:11.645 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:11 smithi045 ceph-mon[30165]: Deploying daemon nvmeof.nvmeof.a on smithi045 2024-05-21T12:24:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:11.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.a", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:24:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.a", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:24:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:11 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:11 smithi092 ceph-mon[33665]: Deploying daemon nvmeof.nvmeof.a on smithi045 2024-05-21T12:24:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.a", "caps": ["mon", "profile rbd", "osd", "profile rbd"]} : dispatch 2024-05-21T12:24:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "client.nvmeof.nvmeof.a", "caps": ["mon", "profile rbd", "osd", "profile rbd"]}]': finished 2024-05-21T12:24:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:11 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:11 smithi072 ceph-mon[33912]: Deploying daemon nvmeof.nvmeof.a on smithi045 2024-05-21T12:24:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:12 smithi045 ceph-mon[30165]: pgmap v248: 33 pgs: 33 active+clean; 582 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:12.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:12 smithi072 ceph-mon[33912]: pgmap v248: 33 pgs: 33 active+clean; 582 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:13.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:12 smithi092 ceph-mon[33665]: pgmap v248: 33 pgs: 33 active+clean; 582 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:14.046 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage14 --size 8192 2024-05-21T12:24:14.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage14 --size 8192 2024-05-21T12:24:14.295 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:14.789 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:14 smithi045 ceph-mon[30165]: pgmap v249: 33 pgs: 33 active+clean; 583 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:14.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:14 smithi072 ceph-mon[33912]: pgmap v249: 33 pgs: 33 active+clean; 583 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:15.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:14 smithi092 ceph-mon[33665]: pgmap v249: 33 pgs: 33 active+clean; 583 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-21T12:24:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:16 smithi045 ceph-mon[30165]: pgmap v250: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-05-21T12:24:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:16 smithi072 ceph-mon[33912]: pgmap v250: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-05-21T12:24:17.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:16 smithi092 ceph-mon[33665]: pgmap v250: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-05-21T12:24:17.441 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage15 --size 8192 2024-05-21T12:24:17.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage15 --size 8192 2024-05-21T12:24:17.758 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:18 smithi045 ceph-mon[30165]: pgmap v251: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-21T12:24:18.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:18 smithi072 ceph-mon[33912]: pgmap v251: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-21T12:24:19.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:18 smithi092 ceph-mon[33665]: pgmap v251: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-21T12:24:20.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:20 smithi092 ceph-mon[33665]: pgmap v252: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:20.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:20 smithi045 ceph-mon[30165]: pgmap v252: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:20.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:20 smithi072 ceph-mon[33912]: pgmap v252: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:22 smithi045 ceph-mon[30165]: pgmap v253: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:22.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:22 smithi072 ceph-mon[33912]: pgmap v253: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:23.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:22 smithi092 ceph-mon[33665]: pgmap v253: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:24 smithi045 ceph-mon[30165]: pgmap v254: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:24.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:24 smithi072 ceph-mon[33912]: pgmap v254: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:25.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:24 smithi092 ceph-mon[33665]: pgmap v254: 33 pgs: 33 active+clean; 584 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:24:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:26 smithi045 ceph-mon[30165]: pgmap v255: 33 pgs: 33 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-21T12:24:26.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:26 smithi072 ceph-mon[33912]: pgmap v255: 33 pgs: 33 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-21T12:24:27.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:26 smithi092 ceph-mon[33665]: pgmap v255: 33 pgs: 33 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-21T12:24:28.296 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage16 --size 8192 2024-05-21T12:24:28.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage16 --size 8192 2024-05-21T12:24:28.788 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:28 smithi045 ceph-mon[30165]: pgmap v256: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-21T12:24:28.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:28 smithi072 ceph-mon[33912]: pgmap v256: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-21T12:24:29.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:28 smithi092 ceph-mon[33665]: pgmap v256: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-21T12:24:30.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:30 smithi045 ceph-mon[30165]: pgmap v257: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 426 B/s wr, 1 op/s 2024-05-21T12:24:30.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:30.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:30.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:30.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:30.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:30 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:24:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:30 smithi072 ceph-mon[33912]: pgmap v257: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 426 B/s wr, 1 op/s 2024-05-21T12:24:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:30.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:30 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:24:31.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:30 smithi092 ceph-mon[33665]: pgmap v257: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 426 B/s wr, 1 op/s 2024-05-21T12:24:31.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:31.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:31.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:31.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:30 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:24:31.723 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:31 smithi045 ceph-mon[30165]: Checking pool "mypool" exists for service nvmeof.mypool 2024-05-21T12:24:31.750 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage17 --size 8192 2024-05-21T12:24:31.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage17 --size 8192 2024-05-21T12:24:31.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:31 smithi072 ceph-mon[33912]: Checking pool "mypool" exists for service nvmeof.mypool 2024-05-21T12:24:32.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:31 smithi092 ceph-mon[33665]: Checking pool "mypool" exists for service nvmeof.mypool 2024-05-21T12:24:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:32 smithi045 ceph-mon[30165]: pgmap v258: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 341 B/s wr, 1 op/s 2024-05-21T12:24:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:32 smithi045 ceph-mon[30165]: mgrmap e15: x(active, since 7m) 2024-05-21T12:24:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:32 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:32 smithi072 ceph-mon[33912]: pgmap v258: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 341 B/s wr, 1 op/s 2024-05-21T12:24:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:32 smithi072 ceph-mon[33912]: mgrmap e15: x(active, since 7m) 2024-05-21T12:24:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:32.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:32 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:33.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:32 smithi092 ceph-mon[33665]: pgmap v258: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 597 B/s rd, 341 B/s wr, 1 op/s 2024-05-21T12:24:33.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:32 smithi092 ceph-mon[33665]: mgrmap e15: x(active, since 7m) 2024-05-21T12:24:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:33.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:32 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: pgmap v259: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 426 B/s wr, 1 op/s 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.a", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.a", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:34.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:34 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.b", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:34.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: pgmap v259: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 426 B/s wr, 1 op/s 2024-05-21T12:24:34.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:34.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:24:34.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.a", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:34.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.a", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:34.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:34 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.b", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: pgmap v259: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 426 B/s wr, 1 op/s 2024-05-21T12:24:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:24:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.a", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.a", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:34.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:34 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.b", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:35.093 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage18 --size 8192 2024-05-21T12:24:35.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage18 --size 8192 2024-05-21T12:24:35.340 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: nvmeof daemon_check_post [(nvmeof.nvmeof.a), (nvmeof.nvmeof.b), (nvmeof.nvmeof.c)] 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: nvmeof daemon_descr nvmeof.nvmeof.a in status running on smithi045 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: nvmeof daemon name=client.nvmeof.nvmeof.a 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.a', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: nvmeof daemon_descr nvmeof.nvmeof.b in status running on smithi072 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: nvmeof daemon name=client.nvmeof.nvmeof.b 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.b', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/586202931' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.b", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:35 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.c", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:35.694 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: nvmeof daemon_check_post [(nvmeof.nvmeof.a), (nvmeof.nvmeof.b), (nvmeof.nvmeof.c)] 2024-05-21T12:24:35.694 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: nvmeof daemon_descr nvmeof.nvmeof.a in status running on smithi045 2024-05-21T12:24:35.694 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: nvmeof daemon name=client.nvmeof.nvmeof.a 2024-05-21T12:24:35.694 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.a', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:35.694 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: nvmeof daemon_descr nvmeof.nvmeof.b in status running on smithi072 2024-05-21T12:24:35.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: nvmeof daemon name=client.nvmeof.nvmeof.b 2024-05-21T12:24:35.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.b', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:35.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/586202931' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:35.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.b", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:35.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:35 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.c", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:35.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: nvmeof daemon_check_post [(nvmeof.nvmeof.a), (nvmeof.nvmeof.b), (nvmeof.nvmeof.c)] 2024-05-21T12:24:35.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: nvmeof daemon_descr nvmeof.nvmeof.a in status running on smithi045 2024-05-21T12:24:35.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: nvmeof daemon name=client.nvmeof.nvmeof.a 2024-05-21T12:24:35.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.a', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:35.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: nvmeof daemon_descr nvmeof.nvmeof.b in status running on smithi072 2024-05-21T12:24:35.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: nvmeof daemon name=client.nvmeof.nvmeof.b 2024-05-21T12:24:35.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.b', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:35.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/586202931' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:35.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.b", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:35.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:35 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.c", "group": "", "pool": "mypool"} : dispatch 2024-05-21T12:24:36.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: pgmap v260: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-21T12:24:36.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: nvmeof daemon_descr nvmeof.nvmeof.c in status running on smithi092 2024-05-21T12:24:36.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: nvmeof daemon name=client.nvmeof.nvmeof.c 2024-05-21T12:24:36.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.c', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:36.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2683154828' entity='client.nvmeof.nvmeof.b' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:36.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.c", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:36.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-list"} : dispatch 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi045"} : dispatch 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi072"} : dispatch 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi092"} : dispatch 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:36.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:24:36.668 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:36 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.711 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: pgmap v260: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-21T12:24:36.711 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: nvmeof daemon_descr nvmeof.nvmeof.c in status running on smithi092 2024-05-21T12:24:36.711 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: nvmeof daemon name=client.nvmeof.nvmeof.c 2024-05-21T12:24:36.711 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.c', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:36.711 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2683154828' entity='client.nvmeof.nvmeof.b' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.c", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-list"} : dispatch 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi045"} : dispatch 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi072"} : dispatch 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi092"} : dispatch 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.712 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:24:36.713 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:36.713 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:24:36.713 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:36 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: pgmap v260: 33 pgs: 33 active+clean; 585 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-21T12:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: nvmeof daemon_descr nvmeof.nvmeof.c in status running on smithi092 2024-05-21T12:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: nvmeof daemon name=client.nvmeof.nvmeof.c 2024-05-21T12:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: create gateway: monitor command {'prefix': 'nvme-gw create', 'id': 'client.nvmeof.nvmeof.c', 'group': '', 'pool': 'mypool'} 2024-05-21T12:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2683154828' entity='client.nvmeof.nvmeof.b' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd='[{"prefix": "nvme-gw create", "id": "client.nvmeof.nvmeof.c", "group": "", "pool": "mypool"}]': finished 2024-05-21T12:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-list"} : dispatch 2024-05-21T12:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi045"} : dispatch 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi072"} : dispatch 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "dashboard nvmeof-gateway-add", "name": "smithi092"} : dispatch 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:24:36.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:24:36.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:36 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:24:37.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:37 smithi045 ceph-mon[30165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-list"}]: dispatch 2024-05-21T12:24:37.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:37 smithi045 ceph-mon[30165]: Adding NVMeoF gateway 172.21.15.45:5500 to Dashboard 2024-05-21T12:24:37.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:37 smithi045 ceph-mon[30165]: Adding NVMeoF gateway 172.21.15.72:5500 to Dashboard 2024-05-21T12:24:37.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:37 smithi045 ceph-mon[30165]: Adding NVMeoF gateway 172.21.15.92:5500 to Dashboard 2024-05-21T12:24:37.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:37 smithi045 ceph-mon[30165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi045"}]: dispatch 2024-05-21T12:24:37.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:37 smithi045 ceph-mon[30165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi072"}]: dispatch 2024-05-21T12:24:37.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:37 smithi045 ceph-mon[30165]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi092"}]: dispatch 2024-05-21T12:24:37.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:37 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/3293786181' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:37.673 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage19 --size 8192 2024-05-21T12:24:37.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage19 --size 8192 2024-05-21T12:24:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:37 smithi092 ceph-mon[33665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-list"}]: dispatch 2024-05-21T12:24:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:37 smithi092 ceph-mon[33665]: Adding NVMeoF gateway 172.21.15.45:5500 to Dashboard 2024-05-21T12:24:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:37 smithi092 ceph-mon[33665]: Adding NVMeoF gateway 172.21.15.72:5500 to Dashboard 2024-05-21T12:24:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:37 smithi092 ceph-mon[33665]: Adding NVMeoF gateway 172.21.15.92:5500 to Dashboard 2024-05-21T12:24:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:37 smithi092 ceph-mon[33665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi045"}]: dispatch 2024-05-21T12:24:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:37 smithi092 ceph-mon[33665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi072"}]: dispatch 2024-05-21T12:24:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:37 smithi092 ceph-mon[33665]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi092"}]: dispatch 2024-05-21T12:24:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:37 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/3293786181' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:37.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:37 smithi072 ceph-mon[33912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-list"}]: dispatch 2024-05-21T12:24:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:37 smithi072 ceph-mon[33912]: Adding NVMeoF gateway 172.21.15.45:5500 to Dashboard 2024-05-21T12:24:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:37 smithi072 ceph-mon[33912]: Adding NVMeoF gateway 172.21.15.72:5500 to Dashboard 2024-05-21T12:24:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:37 smithi072 ceph-mon[33912]: Adding NVMeoF gateway 172.21.15.92:5500 to Dashboard 2024-05-21T12:24:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:37 smithi072 ceph-mon[33912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi045"}]: dispatch 2024-05-21T12:24:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:37 smithi072 ceph-mon[33912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi072"}]: dispatch 2024-05-21T12:24:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:37 smithi072 ceph-mon[33912]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard nvmeof-gateway-add", "name": "smithi092"}]: dispatch 2024-05-21T12:24:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:37 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/3293786181' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:24:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:38.611 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:38 smithi045 ceph-mon[30165]: pgmap v261: 33 pgs: 33 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-21T12:24:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:38 smithi092 ceph-mon[33665]: pgmap v261: 33 pgs: 33 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-21T12:24:38.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:38 smithi072 ceph-mon[33912]: pgmap v261: 33 pgs: 33 active+clean; 585 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-21T12:24:39.878 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage20 --size 8192 2024-05-21T12:24:39.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage20 --size 8192 2024-05-21T12:24:40.123 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:40.542 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:40 smithi045 ceph-mon[30165]: pgmap v262: 33 pgs: 33 active+clean; 586 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-21T12:24:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:40 smithi092 ceph-mon[33665]: pgmap v262: 33 pgs: 33 active+clean; 586 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-21T12:24:40.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:40 smithi072 ceph-mon[33912]: pgmap v262: 33 pgs: 33 active+clean; 586 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-21T12:24:42.153 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage21 --size 8192 2024-05-21T12:24:42.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage21 --size 8192 2024-05-21T12:24:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:42 smithi092 ceph-mon[33665]: pgmap v263: 33 pgs: 33 active+clean; 587 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 853 B/s wr, 3 op/s 2024-05-21T12:24:42.754 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:42 smithi045 ceph-mon[30165]: pgmap v263: 33 pgs: 33 active+clean; 587 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 853 B/s wr, 3 op/s 2024-05-21T12:24:42.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:42 smithi072 ceph-mon[33912]: pgmap v263: 33 pgs: 33 active+clean; 587 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 853 B/s wr, 3 op/s 2024-05-21T12:24:44.520 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage22 --size 8192 2024-05-21T12:24:44.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage22 --size 8192 2024-05-21T12:24:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:44 smithi092 ceph-mon[33665]: pgmap v264: 33 pgs: 33 active+clean; 587 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:24:44.796 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:44.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:44 smithi045 ceph-mon[30165]: pgmap v264: 33 pgs: 33 active+clean; 587 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:24:44.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:44 smithi072 ceph-mon[33912]: pgmap v264: 33 pgs: 33 active+clean; 587 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:24:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:46 smithi092 ceph-mon[33665]: pgmap v265: 33 pgs: 33 active+clean; 588 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:24:46.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:46 smithi045 ceph-mon[30165]: pgmap v265: 33 pgs: 33 active+clean; 588 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:24:46.897 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage23 --size 8192 2024-05-21T12:24:46.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage23 --size 8192 2024-05-21T12:24:46.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:46 smithi072 ceph-mon[33912]: pgmap v265: 33 pgs: 33 active+clean; 588 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:24:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:48.718 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:48 smithi072 ceph-mon[33912]: pgmap v266: 33 pgs: 33 active+clean; 588 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:48 smithi092 ceph-mon[33665]: pgmap v266: 33 pgs: 33 active+clean; 588 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:48 smithi045 ceph-mon[30165]: pgmap v266: 33 pgs: 33 active+clean; 588 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:49.174 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage24 --size 8192 2024-05-21T12:24:49.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage24 --size 8192 2024-05-21T12:24:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:50.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:50 smithi045 ceph-mon[30165]: pgmap v267: 33 pgs: 33 active+clean; 589 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-21T12:24:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:50 smithi092 ceph-mon[33665]: pgmap v267: 33 pgs: 33 active+clean; 589 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-21T12:24:50.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:50 smithi072 ceph-mon[33912]: pgmap v267: 33 pgs: 33 active+clean; 589 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-21T12:24:51.546 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage25 --size 8192 2024-05-21T12:24:51.546 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage25 --size 8192 2024-05-21T12:24:51.796 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:52.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:52 smithi045 ceph-mon[30165]: pgmap v268: 33 pgs: 33 active+clean; 589 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:52 smithi092 ceph-mon[33665]: pgmap v268: 33 pgs: 33 active+clean; 589 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:52.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:52 smithi072 ceph-mon[33912]: pgmap v268: 33 pgs: 33 active+clean; 589 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:54.018 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage26 --size 8192 2024-05-21T12:24:54.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage26 --size 8192 2024-05-21T12:24:54.264 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:24:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:54 smithi092 ceph-mon[33665]: pgmap v269: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:24:54.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:54 smithi045 ceph-mon[30165]: pgmap v269: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:24:54.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:54 smithi072 ceph-mon[33912]: pgmap v269: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:24:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:56 smithi092 ceph-mon[33665]: pgmap v270: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:56.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:56 smithi045 ceph-mon[30165]: pgmap v270: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:56.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:56 smithi072 ceph-mon[33912]: pgmap v270: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:24:58 smithi092 ceph-mon[33665]: pgmap v271: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:58.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:24:58 smithi045 ceph-mon[30165]: pgmap v271: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:24:58.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:24:58 smithi072 ceph-mon[33912]: pgmap v271: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:00 smithi092 ceph-mon[33665]: pgmap v272: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-21T12:25:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:00 smithi045 ceph-mon[30165]: pgmap v272: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-21T12:25:00.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:00 smithi072 ceph-mon[33912]: pgmap v272: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-21T12:25:02.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:02 smithi045 ceph-mon[30165]: pgmap v273: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 938 B/s wr, 4 op/s 2024-05-21T12:25:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:02 smithi092 ceph-mon[33665]: pgmap v273: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 938 B/s wr, 4 op/s 2024-05-21T12:25:02.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:02 smithi072 ceph-mon[33912]: pgmap v273: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 938 B/s wr, 4 op/s 2024-05-21T12:25:03.662 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage27 --size 8192 2024-05-21T12:25:03.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage27 --size 8192 2024-05-21T12:25:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:04 smithi092 ceph-mon[33665]: pgmap v274: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-21T12:25:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:04 smithi045 ceph-mon[30165]: pgmap v274: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-21T12:25:04.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:04 smithi072 ceph-mon[33912]: pgmap v274: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-21T12:25:05.948 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage28 --size 8192 2024-05-21T12:25:05.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage28 --size 8192 2024-05-21T12:25:06.194 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:06.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:06 smithi045 ceph-mon[30165]: pgmap v275: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:25:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:06 smithi092 ceph-mon[33665]: pgmap v275: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:25:06.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:06 smithi072 ceph-mon[33912]: pgmap v275: 33 pgs: 33 active+clean; 590 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:25:08.420 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage29 --size 8192 2024-05-21T12:25:08.421 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage29 --size 8192 2024-05-21T12:25:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:08 smithi092 ceph-mon[33665]: pgmap v276: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-21T12:25:08.790 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:08 smithi045 ceph-mon[30165]: pgmap v276: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-21T12:25:08.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:08 smithi072 ceph-mon[33912]: pgmap v276: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-21T12:25:10.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:10 smithi045 ceph-mon[30165]: pgmap v277: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-21T12:25:10.599 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage30 --size 8192 2024-05-21T12:25:10.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage30 --size 8192 2024-05-21T12:25:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:10 smithi092 ceph-mon[33665]: pgmap v277: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-21T12:25:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:10.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:10 smithi072 ceph-mon[33912]: pgmap v277: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-21T12:25:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:12 smithi092 ceph-mon[33665]: pgmap v278: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 938 B/s wr, 4 op/s 2024-05-21T12:25:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:12 smithi045 ceph-mon[30165]: pgmap v278: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 938 B/s wr, 4 op/s 2024-05-21T12:25:12.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:12 smithi072 ceph-mon[33912]: pgmap v278: 33 pgs: 33 active+clean; 591 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 938 B/s wr, 4 op/s 2024-05-21T12:25:12.981 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage31 --size 8192 2024-05-21T12:25:12.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage31 --size 8192 2024-05-21T12:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:14.696 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:14 smithi045 ceph-mon[30165]: pgmap v279: 33 pgs: 33 active+clean; 592 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:14 smithi092 ceph-mon[33665]: pgmap v279: 33 pgs: 33 active+clean; 592 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:14.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:14 smithi072 ceph-mon[33912]: pgmap v279: 33 pgs: 33 active+clean; 592 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:15.322 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage32 --size 8192 2024-05-21T12:25:15.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage32 --size 8192 2024-05-21T12:25:15.581 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:16.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:16 smithi045 ceph-mon[30165]: pgmap v280: 33 pgs: 33 active+clean; 592 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:16 smithi092 ceph-mon[33665]: pgmap v280: 33 pgs: 33 active+clean; 592 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:16 smithi072 ceph-mon[33912]: pgmap v280: 33 pgs: 33 active+clean; 592 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:17.733 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage33 --size 8192 2024-05-21T12:25:17.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage33 --size 8192 2024-05-21T12:25:17.983 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:18.722 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:18 smithi072 ceph-mon[33912]: pgmap v281: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:25:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:18 smithi092 ceph-mon[33665]: pgmap v281: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:25:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:18 smithi045 ceph-mon[30165]: pgmap v281: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:25:20.262 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage34 --size 8192 2024-05-21T12:25:20.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage34 --size 8192 2024-05-21T12:25:20.508 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:20 smithi045 ceph-mon[30165]: pgmap v282: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:20.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:20 smithi092 ceph-mon[33665]: pgmap v282: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:20.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:20 smithi072 ceph-mon[33912]: pgmap v282: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:22.718 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:22 smithi045 ceph-mon[30165]: pgmap v283: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:22.745 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage35 --size 8192 2024-05-21T12:25:22.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage35 --size 8192 2024-05-21T12:25:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:22 smithi092 ceph-mon[33665]: pgmap v283: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:22.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:22 smithi072 ceph-mon[33912]: pgmap v283: 33 pgs: 33 active+clean; 594 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:24 smithi092 ceph-mon[33665]: pgmap v284: 33 pgs: 33 active+clean; 595 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-21T12:25:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:24 smithi045 ceph-mon[30165]: pgmap v284: 33 pgs: 33 active+clean; 595 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-21T12:25:24.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:24 smithi072 ceph-mon[33912]: pgmap v284: 33 pgs: 33 active+clean; 595 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-21T12:25:25.176 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage36 --size 8192 2024-05-21T12:25:25.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage36 --size 8192 2024-05-21T12:25:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:26.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:26 smithi045 ceph-mon[30165]: pgmap v285: 33 pgs: 33 active+clean; 595 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:26 smithi092 ceph-mon[33665]: pgmap v285: 33 pgs: 33 active+clean; 595 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:26.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:26 smithi072 ceph-mon[33912]: pgmap v285: 33 pgs: 33 active+clean; 595 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:27.556 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage37 --size 8192 2024-05-21T12:25:27.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage37 --size 8192 2024-05-21T12:25:27.805 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:28 smithi092 ceph-mon[33665]: pgmap v286: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:25:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:28 smithi045 ceph-mon[30165]: pgmap v286: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:25:28.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:28 smithi072 ceph-mon[33912]: pgmap v286: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:25:29.880 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage38 --size 8192 2024-05-21T12:25:29.880 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage38 --size 8192 2024-05-21T12:25:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:30.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:30 smithi045 ceph-mon[30165]: pgmap v287: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:30 smithi092 ceph-mon[33665]: pgmap v287: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:30.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:30 smithi072 ceph-mon[33912]: pgmap v287: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:32.235 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage39 --size 8192 2024-05-21T12:25:32.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage39 --size 8192 2024-05-21T12:25:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:32 smithi092 ceph-mon[33665]: pgmap v288: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:32.786 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:32 smithi045 ceph-mon[30165]: pgmap v288: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:32.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:32 smithi072 ceph-mon[33912]: pgmap v288: 33 pgs: 33 active+clean; 596 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:34.614 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage40 --size 8192 2024-05-21T12:25:34.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage40 --size 8192 2024-05-21T12:25:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:34 smithi092 ceph-mon[33665]: pgmap v289: 33 pgs: 33 active+clean; 597 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:25:34.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:34 smithi045 ceph-mon[30165]: pgmap v289: 33 pgs: 33 active+clean; 597 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:25:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:34.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:34 smithi072 ceph-mon[33912]: pgmap v289: 33 pgs: 33 active+clean; 597 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-21T12:25:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:36 smithi092 ceph-mon[33665]: pgmap v290: 33 pgs: 33 active+clean; 597 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:36.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:36 smithi045 ceph-mon[30165]: pgmap v290: 33 pgs: 33 active+clean; 597 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:36.831 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage41 --size 8192 2024-05-21T12:25:36.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage41 --size 8192 2024-05-21T12:25:36.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:36 smithi072 ceph-mon[33912]: pgmap v290: 33 pgs: 33 active+clean; 597 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:25:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:25:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:25:37.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:37 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:25:37.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:25:37.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:25:37.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:25:37.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:37 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:25:37.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:25:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:25:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:25:37.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:37 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:25:38.533 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:38 smithi045 ceph-mon[30165]: pgmap v291: 33 pgs: 33 active+clean; 598 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-21T12:25:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:38 smithi092 ceph-mon[33665]: pgmap v291: 33 pgs: 33 active+clean; 598 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-21T12:25:38.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:38 smithi072 ceph-mon[33912]: pgmap v291: 33 pgs: 33 active+clean; 598 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-21T12:25:39.093 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage42 --size 8192 2024-05-21T12:25:39.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage42 --size 8192 2024-05-21T12:25:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:40.618 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:40 smithi045 ceph-mon[30165]: pgmap v292: 33 pgs: 33 active+clean; 598 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:40 smithi092 ceph-mon[33665]: pgmap v292: 33 pgs: 33 active+clean; 598 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:40.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:40 smithi072 ceph-mon[33912]: pgmap v292: 33 pgs: 33 active+clean; 598 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:41.374 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage43 --size 8192 2024-05-21T12:25:41.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage43 --size 8192 2024-05-21T12:25:41.630 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:42.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:42 smithi045 ceph-mon[30165]: pgmap v293: 33 pgs: 33 active+clean; 599 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:42 smithi092 ceph-mon[33665]: pgmap v293: 33 pgs: 33 active+clean; 599 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:42.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:42 smithi072 ceph-mon[33912]: pgmap v293: 33 pgs: 33 active+clean; 599 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:43.738 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage44 --size 8192 2024-05-21T12:25:43.738 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage44 --size 8192 2024-05-21T12:25:43.986 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:44 smithi092 ceph-mon[33665]: pgmap v294: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:25:44.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:44 smithi045 ceph-mon[30165]: pgmap v294: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:25:44.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:44 smithi072 ceph-mon[33912]: pgmap v294: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:25:46.237 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage45 --size 8192 2024-05-21T12:25:46.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage45 --size 8192 2024-05-21T12:25:46.493 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:46.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:46 smithi045 ceph-mon[30165]: pgmap v295: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:46 smithi092 ceph-mon[33665]: pgmap v295: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:46.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:46 smithi072 ceph-mon[33912]: pgmap v295: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:48.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:48 smithi045 ceph-mon[30165]: pgmap v296: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:48.642 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage46 --size 8192 2024-05-21T12:25:48.643 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage46 --size 8192 2024-05-21T12:25:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:48 smithi092 ceph-mon[33665]: pgmap v296: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:48.895 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:48.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:48 smithi072 ceph-mon[33912]: pgmap v296: 33 pgs: 33 active+clean; 600 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:25:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:50 smithi092 ceph-mon[33665]: pgmap v297: 33 pgs: 33 active+clean; 601 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:50 smithi045 ceph-mon[30165]: pgmap v297: 33 pgs: 33 active+clean; 601 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:50 smithi072 ceph-mon[33912]: pgmap v297: 33 pgs: 33 active+clean; 601 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:25:51.087 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage47 --size 8192 2024-05-21T12:25:51.088 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage47 --size 8192 2024-05-21T12:25:51.335 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:52.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:52 smithi045 ceph-mon[30165]: pgmap v298: 33 pgs: 33 active+clean; 601 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:52 smithi092 ceph-mon[33665]: pgmap v298: 33 pgs: 33 active+clean; 601 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:52.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:52 smithi072 ceph-mon[33912]: pgmap v298: 33 pgs: 33 active+clean; 601 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:25:53.478 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage48 --size 8192 2024-05-21T12:25:53.478 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage48 --size 8192 2024-05-21T12:25:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:54 smithi092 ceph-mon[33665]: pgmap v299: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:25:54.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:54 smithi045 ceph-mon[30165]: pgmap v299: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:25:54.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:54 smithi072 ceph-mon[33912]: pgmap v299: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:25:55.874 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage49 --size 8192 2024-05-21T12:25:55.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage49 --size 8192 2024-05-21T12:25:56.120 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:56 smithi092 ceph-mon[33665]: pgmap v300: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:56.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:56 smithi045 ceph-mon[30165]: pgmap v300: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:56.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:56 smithi072 ceph-mon[33912]: pgmap v300: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:25:58.104 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage50 --size 8192 2024-05-21T12:25:58.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage50 --size 8192 2024-05-21T12:25:58.352 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:25:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:25:58 smithi092 ceph-mon[33665]: pgmap v301: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-21T12:25:58.772 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:25:58 smithi045 ceph-mon[30165]: pgmap v301: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-21T12:25:58.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:25:58 smithi072 ceph-mon[33912]: pgmap v301: 33 pgs: 33 active+clean; 602 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-21T12:26:00.624 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage51 --size 8192 2024-05-21T12:26:00.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage51 --size 8192 2024-05-21T12:26:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:00 smithi092 ceph-mon[33665]: pgmap v302: 33 pgs: 33 active+clean; 603 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:26:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:00 smithi045 ceph-mon[30165]: pgmap v302: 33 pgs: 33 active+clean; 603 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:26:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:00.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:00 smithi072 ceph-mon[33912]: pgmap v302: 33 pgs: 33 active+clean; 603 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:26:02.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:02 smithi045 ceph-mon[30165]: pgmap v303: 33 pgs: 33 active+clean; 603 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:26:02.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:02 smithi072 ceph-mon[33912]: pgmap v303: 33 pgs: 33 active+clean; 603 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:26:02.931 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage52 --size 8192 2024-05-21T12:26:02.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage52 --size 8192 2024-05-21T12:26:03.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:02 smithi092 ceph-mon[33665]: pgmap v303: 33 pgs: 33 active+clean; 603 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:26:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:04.523 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:04 smithi045 ceph-mon[30165]: pgmap v304: 33 pgs: 33 active+clean; 604 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:26:04.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:04 smithi072 ceph-mon[33912]: pgmap v304: 33 pgs: 33 active+clean; 604 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:26:05.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:04 smithi092 ceph-mon[33665]: pgmap v304: 33 pgs: 33 active+clean; 604 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-21T12:26:05.520 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage53 --size 8192 2024-05-21T12:26:05.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage53 --size 8192 2024-05-21T12:26:05.769 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:06 smithi045 ceph-mon[30165]: pgmap v305: 33 pgs: 33 active+clean; 604 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:06.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:06 smithi072 ceph-mon[33912]: pgmap v305: 33 pgs: 33 active+clean; 604 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:07.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:06 smithi092 ceph-mon[33665]: pgmap v305: 33 pgs: 33 active+clean; 604 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:07.903 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage54 --size 8192 2024-05-21T12:26:07.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage54 --size 8192 2024-05-21T12:26:08.149 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:08.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:08 smithi045 ceph-mon[30165]: pgmap v306: 33 pgs: 33 active+clean; 605 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:26:08.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:08 smithi072 ceph-mon[33912]: pgmap v306: 33 pgs: 33 active+clean; 605 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:26:09.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:08 smithi092 ceph-mon[33665]: pgmap v306: 33 pgs: 33 active+clean; 605 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:26:10.334 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage55 --size 8192 2024-05-21T12:26:10.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage55 --size 8192 2024-05-21T12:26:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:10 smithi045 ceph-mon[30165]: pgmap v307: 33 pgs: 33 active+clean; 606 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-21T12:26:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:10.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:10 smithi072 ceph-mon[33912]: pgmap v307: 33 pgs: 33 active+clean; 606 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-21T12:26:11.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:10 smithi092 ceph-mon[33665]: pgmap v307: 33 pgs: 33 active+clean; 606 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-21T12:26:12.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:12 smithi045 ceph-mon[30165]: pgmap v308: 33 pgs: 33 active+clean; 606 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:12.741 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage56 --size 8192 2024-05-21T12:26:12.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage56 --size 8192 2024-05-21T12:26:12.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:12 smithi072 ceph-mon[33912]: pgmap v308: 33 pgs: 33 active+clean; 606 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:12.987 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:13.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:12 smithi092 ceph-mon[33665]: pgmap v308: 33 pgs: 33 active+clean; 606 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:14.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:14 smithi045 ceph-mon[30165]: pgmap v309: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:26:14.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:14 smithi072 ceph-mon[33912]: pgmap v309: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:26:15.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:14 smithi092 ceph-mon[33665]: pgmap v309: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:26:15.080 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage57 --size 8192 2024-05-21T12:26:15.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage57 --size 8192 2024-05-21T12:26:15.332 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:16.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:16 smithi045 ceph-mon[30165]: pgmap v310: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-21T12:26:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:16 smithi072 ceph-mon[33912]: pgmap v310: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-21T12:26:17.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:16 smithi092 ceph-mon[33665]: pgmap v310: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-21T12:26:17.554 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage58 --size 8192 2024-05-21T12:26:17.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage58 --size 8192 2024-05-21T12:26:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:18.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:18 smithi045 ceph-mon[30165]: pgmap v311: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:26:18.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:18 smithi072 ceph-mon[33912]: pgmap v311: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:26:19.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:18 smithi092 ceph-mon[33665]: pgmap v311: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-21T12:26:19.933 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage59 --size 8192 2024-05-21T12:26:19.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage59 --size 8192 2024-05-21T12:26:20.180 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:20 smithi045 ceph-mon[30165]: pgmap v312: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-21T12:26:20.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:20 smithi092 ceph-mon[33665]: pgmap v312: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-21T12:26:20.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:20 smithi072 ceph-mon[33912]: pgmap v312: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-21T12:26:22.466 INFO:tasks.nvmeof:[nvmeof]: rbd create mypool/myimage60 --size 8192 2024-05-21T12:26:22.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- rbd create mypool/myimage60 --size 8192 2024-05-21T12:26:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:22 smithi045 ceph-mon[30165]: pgmap v313: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:22.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:22 smithi072 ceph-mon[33912]: pgmap v313: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:23.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:22 smithi092 ceph-mon[33665]: pgmap v313: 33 pgs: 33 active+clean; 607 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-21T12:26:24.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:23 smithi045 ceph-mon[30165]: pgmap v314: 33 pgs: 33 active+clean; 609 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:26:24.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:23 smithi092 ceph-mon[33665]: pgmap v314: 33 pgs: 33 active+clean; 609 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:26:24.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:23 smithi072 ceph-mon[33912]: pgmap v314: 33 pgs: 33 active+clean; 609 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T12:26:24.813 DEBUG:teuthology.orchestra.run.smithi045:nvmeof.nvmeof.a> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:26:24.816 DEBUG:teuthology.orchestra.run.smithi072:nvmeof.nvmeof.b> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:26:24.818 DEBUG:teuthology.orchestra.run.smithi092:nvmeof.nvmeof.c> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:26:24.820 INFO:tasks.nvmeof:[nvmeof]: executed deploy_nvmeof successfully! 2024-05-21T12:26:24.821 INFO:tasks.nvmeof:[nvmeof]: running set_gateway_cfg... 2024-05-21T12:26:24.821 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-21T12:26:24.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/nvmeof.env 2024-05-21T12:26:24.882 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-21T12:26:24.882 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/nvmeof.env 2024-05-21T12:26:24.907 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-21T12:26:24.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/nvmeof.env 2024-05-21T12:26:24.932 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:26:24.933 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/nvmeof.env 2024-05-21T12:26:24.965 INFO:tasks.nvmeof:[nvmeof]: executed set_gateway_cfg successfully! 2024-05-21T12:26:24.966 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2024-05-21T12:26:24.976 INFO:tasks.cephadm:Waiting for ceph service nvmeof.mypool to start (timeout 300)... 2024-05-21T12:26:24.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 -- ceph orch ls -f json 2024-05-21T12:26:25.220 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T12:26:26.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:26 smithi045 ceph-mon[30165]: pgmap v315: 33 pgs: 33 active+clean; 609 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:26:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:26 smithi072 ceph-mon[33912]: pgmap v315: 33 pgs: 33 active+clean; 609 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:26:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:26 smithi092 ceph-mon[33665]: pgmap v315: 33 pgs: 33 active+clean; 609 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-21T12:26:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:26:27.095 INFO:teuthology.orchestra.run.smithi045.stdout:[{"events": ["2024-05-21T12:18:42.414502Z service:mgr [INFO] \"service was created\""], "placement": {"count": 1, "hosts": ["smithi045=x"]}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-05-21T12:18:41.972146Z", "last_refresh": "2024-05-21T12:24:33.402870Z", "running": 1, "size": 1}}, {"events": ["2024-05-21T12:18:35.019872Z service:mon [INFO] \"service was created\""], "placement": {"count": 3, "hosts": ["smithi045:172.21.15.45=a", "smithi072:172.21.15.72=b", "smithi092:172.21.15.92=c"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-05-21T12:18:16.727847Z", "last_refresh": "2024-05-21T12:24:31.895544Z", "running": 3, "size": 3}}, {"events": ["2024-05-21T12:24:30.171252Z service:nvmeof.mypool [INFO] \"service was created\""], "placement": {"count": 3, "hosts": ["smithi045=nvmeof.a", "smithi072=nvmeof.b", "smithi092=nvmeof.c"]}, "service_id": "mypool", "service_name": "nvmeof.mypool", "service_type": "nvmeof", "spec": {"allowed_consecutive_spdk_ping_failures": 1, "bdevs_per_cluster": 32, "client_cert": "./client.crt", "client_key": "./client.key", "conn_retries": 10, "discovery_port": 8009, "enable_monitor_client": true, "enable_prometheus_exporter": true, "log_directory": "/var/log/ceph/", "log_files_enabled": true, "log_files_rotation_enabled": true, "log_level": "INFO", "max_log_directory_backups": 10, "max_log_file_size_in_mb": 10, "max_log_files_count": 20, "monitor_timeout": 1.0, "omap_file_lock_duration": 20, "omap_file_lock_retries": 30, "omap_file_lock_retry_sleep_interval": 1.0, "omap_file_update_reloads": 10, "pool": "mypool", "port": 5500, "rpc_socket_dir": "/var/tmp/", "rpc_socket_name": "spdk.sock", "server_cert": "./server.crt", "server_key": "./server.key", "spdk_log_level": "WARNING", "spdk_path": "/usr/local/bin/nvmf_tgt", "spdk_ping_interval_in_seconds": 2.0, "spdk_timeout": 60.0, "state_update_interval_sec": 5, "state_update_notify": true, "tgt_path": "/usr/local/bin/nvmf_tgt", "transport_tcp_options": {"in_capsule_data_size": 8192, "max_io_qpairs_per_ctrlr": 7}, "transports": "tcp", "verbose_log_messages": true, "verify_nqns": true}, "status": {"created": "2024-05-21T12:23:40.477585Z", "last_refresh": "2024-05-21T12:24:31.895898Z", "ports": [5500, 4420, 8009], "running": 3, "size": 3}}, {"service_name": "osd", "service_type": "osd", "spec": {"filter_logic": "AND", "objectstore": "bluestore"}, "status": {"container_image_id": "1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815", "container_image_name": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93", "last_refresh": "2024-05-21T12:24:31.895652Z", "running": 8, "size": 8}, "unmanaged": true}] 2024-05-21T12:26:27.702 INFO:tasks.cephadm:nvmeof.mypool has 3/3 2024-05-21T12:26:27.703 INFO:teuthology.run_tasks:Running task workunit... 2024-05-21T12:26:27.719 INFO:tasks.workunit:Pulling workunits from ref 68fa3266016634745d991c9967c479c757966afe 2024-05-21T12:26:27.719 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-21T12:26:27.719 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.3 2024-05-21T12:26:27.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:26:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-21T12:26:27.739 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -- /home/ubuntu/cephtest/mnt.3 2024-05-21T12:26:27.797 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.3 2024-05-21T12:26:27.797 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.3 && mkdir -- client.3 2024-05-21T12:26:27.855 INFO:tasks.workunit:timeout=3h 2024-05-21T12:26:27.855 INFO:tasks.workunit:cleanup=True 2024-05-21T12:26:27.856 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.3 && git clone https://github.com/VallariAg/ceph /home/ubuntu/cephtest/clone.client.3 && cd /home/ubuntu/cephtest/clone.client.3 && git checkout 68fa3266016634745d991c9967c479c757966afe 2024-05-21T12:26:27.913 INFO:tasks.workunit.client.3.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.3'... 2024-05-21T12:26:28.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:28 smithi072 ceph-mon[33912]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:26:28.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:28 smithi072 ceph-mon[33912]: pgmap v316: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-21T12:26:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:28 smithi092 ceph-mon[33665]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:26:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:28 smithi092 ceph-mon[33665]: pgmap v316: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-21T12:26:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:28 smithi045 ceph-mon[30165]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T12:26:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:28 smithi045 ceph-mon[30165]: pgmap v316: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-21T12:26:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:30 smithi072 ceph-mon[33912]: pgmap v317: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-21T12:26:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:30 smithi092 ceph-mon[33665]: pgmap v317: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-21T12:26:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:30 smithi045 ceph-mon[30165]: pgmap v317: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-05-21T12:26:32.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:32 smithi072 ceph-mon[33912]: pgmap v318: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-05-21T12:26:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:32 smithi092 ceph-mon[33665]: pgmap v318: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-05-21T12:26:32.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:32 smithi045 ceph-mon[30165]: pgmap v318: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-05-21T12:26:34.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:34 smithi072 ceph-mon[33912]: pgmap v319: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-21T12:26:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:34 smithi092 ceph-mon[33665]: pgmap v319: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-21T12:26:34.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:34 smithi045 ceph-mon[30165]: pgmap v319: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-21T12:26:36.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:36 smithi072 ceph-mon[33912]: pgmap v320: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-21T12:26:36.760 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:36 smithi092 ceph-mon[33665]: pgmap v320: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-21T12:26:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:36 smithi045 ceph-mon[30165]: pgmap v320: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-21T12:26:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:38 smithi072 ceph-mon[33912]: pgmap v321: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-21T12:26:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:26:38.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:26:38.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:26:38.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:38 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:26:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:38 smithi092 ceph-mon[33665]: pgmap v321: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-21T12:26:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:26:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:26:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:26:38.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:38 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:26:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:38 smithi045 ceph-mon[30165]: pgmap v321: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-05-21T12:26:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:26:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:26:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:26:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:38 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:26:40.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:40 smithi072 ceph-mon[33912]: pgmap v322: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-21T12:26:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:40 smithi092 ceph-mon[33665]: pgmap v322: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-21T12:26:40.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:40 smithi045 ceph-mon[30165]: pgmap v322: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-21T12:26:42.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:42 smithi072 ceph-mon[33912]: pgmap v323: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:42 smithi092 ceph-mon[33665]: pgmap v323: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:42.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:42 smithi045 ceph-mon[30165]: pgmap v323: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:44.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:44 smithi072 ceph-mon[33912]: pgmap v324: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:44 smithi092 ceph-mon[33665]: pgmap v324: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:44.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:44 smithi045 ceph-mon[30165]: pgmap v324: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:46.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:46 smithi072 ceph-mon[33912]: pgmap v325: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:46 smithi092 ceph-mon[33665]: pgmap v325: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:46.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:46 smithi045 ceph-mon[30165]: pgmap v325: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:48.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:48 smithi072 ceph-mon[33912]: pgmap v326: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:48 smithi092 ceph-mon[33665]: pgmap v326: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:48 smithi045 ceph-mon[30165]: pgmap v326: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:50.618 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:50 smithi045 ceph-mon[30165]: pgmap v327: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:50.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:50 smithi072 ceph-mon[33912]: pgmap v327: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:50.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:50 smithi092 ceph-mon[33665]: pgmap v327: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:52.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:52 smithi072 ceph-mon[33912]: pgmap v328: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:52 smithi092 ceph-mon[33665]: pgmap v328: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:52.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:52 smithi045 ceph-mon[30165]: pgmap v328: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:54.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:54 smithi072 ceph-mon[33912]: pgmap v329: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:54 smithi092 ceph-mon[33665]: pgmap v329: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:54.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:54 smithi045 ceph-mon[30165]: pgmap v329: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:56.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:56 smithi072 ceph-mon[33912]: pgmap v330: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:56 smithi092 ceph-mon[33665]: pgmap v330: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:56.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:56 smithi045 ceph-mon[30165]: pgmap v330: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:26:58.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:26:58 smithi072 ceph-mon[33912]: pgmap v331: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:26:58 smithi092 ceph-mon[33665]: pgmap v331: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:26:58.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:26:58 smithi045 ceph-mon[30165]: pgmap v331: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:00.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:00 smithi072 ceph-mon[33912]: pgmap v332: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:00 smithi092 ceph-mon[33665]: pgmap v332: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:00 smithi045 ceph-mon[30165]: pgmap v332: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:02.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:02 smithi072 ceph-mon[33912]: pgmap v333: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:02 smithi092 ceph-mon[33665]: pgmap v333: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:02.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:02 smithi045 ceph-mon[30165]: pgmap v333: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:04.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:04 smithi072 ceph-mon[33912]: pgmap v334: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:04 smithi092 ceph-mon[33665]: pgmap v334: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:04 smithi045 ceph-mon[30165]: pgmap v334: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:06 smithi072 ceph-mon[33912]: pgmap v335: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:06 smithi092 ceph-mon[33665]: pgmap v335: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:06 smithi045 ceph-mon[30165]: pgmap v335: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:08.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:08 smithi072 ceph-mon[33912]: pgmap v336: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:08 smithi092 ceph-mon[33665]: pgmap v336: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:08 smithi045 ceph-mon[30165]: pgmap v336: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:10 smithi072 ceph-mon[33912]: pgmap v337: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:10 smithi092 ceph-mon[33665]: pgmap v337: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:10 smithi045 ceph-mon[30165]: pgmap v337: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:12 smithi072 ceph-mon[33912]: pgmap v338: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:12 smithi092 ceph-mon[33665]: pgmap v338: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:12 smithi045 ceph-mon[30165]: pgmap v338: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:14 smithi072 ceph-mon[33912]: pgmap v339: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:14 smithi092 ceph-mon[33665]: pgmap v339: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:14 smithi045 ceph-mon[30165]: pgmap v339: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:16 smithi072 ceph-mon[33912]: pgmap v340: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:16 smithi092 ceph-mon[33665]: pgmap v340: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:16 smithi045 ceph-mon[30165]: pgmap v340: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:18.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:18 smithi072 ceph-mon[33912]: pgmap v341: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:18 smithi092 ceph-mon[33665]: pgmap v341: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:18 smithi045 ceph-mon[30165]: pgmap v341: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:20.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:20 smithi072 ceph-mon[33912]: pgmap v342: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:20.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:20 smithi092 ceph-mon[33665]: pgmap v342: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:20.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:20 smithi045 ceph-mon[30165]: pgmap v342: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:22 smithi072 ceph-mon[33912]: pgmap v343: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:22 smithi092 ceph-mon[33665]: pgmap v343: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:22 smithi045 ceph-mon[30165]: pgmap v343: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:24.528 INFO:tasks.workunit.client.3.smithi137.stderr:Note: switching to '68fa3266016634745d991c9967c479c757966afe'. 2024-05-21T12:27:24.528 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:27:24.528 INFO:tasks.workunit.client.3.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-21T12:27:24.528 INFO:tasks.workunit.client.3.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-21T12:27:24.528 INFO:tasks.workunit.client.3.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-05-21T12:27:24.528 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr: git switch -c 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr:Or undo this operation with: 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr: git switch - 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:27:24.529 INFO:tasks.workunit.client.3.smithi137.stderr:HEAD is now at 68fa3266016 restructure into "basic" and "thrash" subsuites 2024-05-21T12:27:24.533 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.3/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.3 2024-05-21T12:27:24.621 INFO:tasks.workunit.client.3.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-21T12:27:24.623 INFO:tasks.workunit.client.3.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.3/qa/workunits/direct_io' 2024-05-21T12:27:24.623 INFO:tasks.workunit.client.3.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-21T12:27:24.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:24 smithi072 ceph-mon[33912]: pgmap v344: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:24 smithi092 ceph-mon[33665]: pgmap v344: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:24 smithi045 ceph-mon[30165]: pgmap v344: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:25.313 INFO:tasks.workunit.client.3.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-21T12:27:25.374 INFO:tasks.workunit.client.3.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-21T12:27:25.414 INFO:tasks.workunit.client.3.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.3/qa/workunits/direct_io' 2024-05-21T12:27:25.416 INFO:tasks.workunit.client.3.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.3/qa/workunits/fs' 2024-05-21T12:27:25.416 INFO:tasks.workunit.client.3.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-21T12:27:25.455 INFO:tasks.workunit.client.3.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.3/qa/workunits/fs' 2024-05-21T12:27:25.458 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:27:25.458 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.3 of=/dev/stdout 2024-05-21T12:27:25.475 INFO:tasks.workunit:Running workunits matching rbd/nvmeof_setup_subsystem.sh on client.3... 2024-05-21T12:27:25.476 INFO:tasks.workunit:Running workunit rbd/nvmeof_setup_subsystem.sh... 2024-05-21T12:27:25.477 DEBUG:teuthology.orchestra.run.smithi137:workunit test rbd/nvmeof_setup_subsystem.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && cd -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68fa3266016634745d991c9967c479c757966afe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="3" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.3 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.3 CEPH_MNT=/home/ubuntu/cephtest/mnt.3 RBD_IMAGE_PREFIX=myimage RBD_POOL=mypool timeout 3h /home/ubuntu/cephtest/clone.client.3/qa/workunits/rbd/nvmeof_setup_subsystem.sh 2024-05-21T12:27:25.532 INFO:tasks.workunit.client.3.smithi137.stderr:+ source /etc/ceph/nvmeof.env 2024-05-21T12:27:25.532 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_GATEWAY_IP_ADDRESSES=172.21.15.45,172.21.15.72,172.21.15.92 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_GATEWAY_NAMES=smithi045,smithi072,smithi092 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_DEFAULT_GATEWAY_IP_ADDRESS=172.21.15.45 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_CLI_IMAGE=quay.io/ceph/nvmeof-cli:1.2 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_SUBSYSTEMS_PREFIX=nqn.2016-06.io.spdk:cnode 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_SUBSYSTEMS_COUNT=3 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_NAMESPACES_COUNT=20 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_PORT=4420 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_SRPORT=5500 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:+ RBD_POOL=mypool 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:+ RBD_IMAGE_PREFIX=myimage 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:++ hostname 2024-05-21T12:27:25.533 INFO:tasks.workunit.client.3.smithi137.stderr:+ HOSTNAME=smithi137 2024-05-21T12:27:25.534 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman images 2024-05-21T12:27:25.564 INFO:tasks.workunit.client.3.smithi137.stdout:REPOSITORY TAG IMAGE ID CREATED SIZE 2024-05-21T12:27:25.564 INFO:tasks.workunit.client.3.smithi137.stdout:quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph b0c764b6e22fbb51f59e469d9dd99895c152f73e 1238c5f3fda9 5 days ago 1.44 GB 2024-05-21T12:27:25.567 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman ps 2024-05-21T12:27:25.595 INFO:tasks.workunit.client.3.smithi137.stdout:CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 2024-05-21T12:27:25.598 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 --format json subsystem list 2024-05-21T12:27:25.627 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:25Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:25.707 INFO:tasks.workunit.client.3.smithi137.stderr:Trying to pull quay.io/ceph/nvmeof-cli:1.2... 2024-05-21T12:27:26.203 INFO:tasks.workunit.client.3.smithi137.stderr:Getting image source signatures 2024-05-21T12:27:26.203 INFO:tasks.workunit.client.3.smithi137.stderr:Copying blob sha256:bbbd94e2f9c332f454dee04bf48ca2c56cea0ee9c1986b648148c8beeb8823a8 2024-05-21T12:27:26.203 INFO:tasks.workunit.client.3.smithi137.stderr:Copying blob sha256:4a49477c8b567a11b232542ca112af93d3a9c0f9059cd6c4ef8210412a1dcddf 2024-05-21T12:27:26.203 INFO:tasks.workunit.client.3.smithi137.stderr:Copying blob sha256:99a7ec374c4d4af5f9b9ad281068affa7961aad432aba4efc963785a5a350f8d 2024-05-21T12:27:26.203 INFO:tasks.workunit.client.3.smithi137.stderr:Copying blob sha256:4085cf55ce5819ec62694e99a4085836790644c3ddc2d1295baa2a2f372752b2 2024-05-21T12:27:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:26 smithi072 ceph-mon[33912]: pgmap v345: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:26 smithi092 ceph-mon[33665]: pgmap v345: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:26 smithi045 ceph-mon[30165]: pgmap v345: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:28.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:28 smithi072 ceph-mon[33912]: pgmap v346: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:28 smithi092 ceph-mon[33665]: pgmap v346: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:28 smithi045 ceph-mon[30165]: pgmap v346: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:30 smithi072 ceph-mon[33912]: pgmap v347: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:30 smithi092 ceph-mon[33665]: pgmap v347: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:30 smithi045 ceph-mon[30165]: pgmap v347: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:32.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:32 smithi072 ceph-mon[33912]: pgmap v348: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:32 smithi092 ceph-mon[33665]: pgmap v348: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:32.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:32 smithi045 ceph-mon[30165]: pgmap v348: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:34.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:34 smithi072 ceph-mon[33912]: pgmap v349: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:34 smithi092 ceph-mon[33665]: pgmap v349: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:34.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:34 smithi045 ceph-mon[30165]: pgmap v349: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:35.314 INFO:tasks.workunit.client.3.smithi137.stderr:Copying config sha256:6920e901e334b9921b5bb685a0f626ddb4089728bfd2943b7ebde49fbff025ed 2024-05-21T12:27:35.490 INFO:tasks.workunit.client.3.smithi137.stderr:Writing manifest to image destination 2024-05-21T12:27:36.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:36 smithi072 ceph-mon[33912]: pgmap v350: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:36 smithi092 ceph-mon[33665]: pgmap v350: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:36 smithi045 ceph-mon[30165]: pgmap v350: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:36.978 INFO:tasks.workunit.client.3.smithi137.stdout:{ 2024-05-21T12:27:36.978 INFO:tasks.workunit.client.3.smithi137.stdout: "error_message": "Success", 2024-05-21T12:27:36.978 INFO:tasks.workunit.client.3.smithi137.stdout: "status": 0, 2024-05-21T12:27:36.978 INFO:tasks.workunit.client.3.smithi137.stdout: "subsystems": [] 2024-05-21T12:27:36.978 INFO:tasks.workunit.client.3.smithi137.stdout:} 2024-05-21T12:27:37.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:36] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:52636 2024-05-21T12:27:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:38 smithi072 ceph-mon[33912]: pgmap v351: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:38 smithi092 ceph-mon[33665]: pgmap v351: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:38 smithi045 ceph-mon[30165]: pgmap v351: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:39.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:27:39.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:39.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:39 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:39.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:27:39.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:39.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:39 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:39.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:27:39.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:39.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:39 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:40.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:40 smithi072 ceph-mon[33912]: pgmap v352: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:40 smithi092 ceph-mon[33665]: pgmap v352: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:40.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:40 smithi045 ceph-mon[30165]: pgmap v352: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:42.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:42 smithi072 ceph-mon[33912]: pgmap v353: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:42 smithi092 ceph-mon[33665]: pgmap v353: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:42.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:42 smithi045 ceph-mon[30165]: pgmap v353: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:43.066 INFO:tasks.workunit.client.3.smithi137.stderr:+ IFS=, 2024-05-21T12:27:43.066 INFO:tasks.workunit.client.3.smithi137.stderr:+ read -ra gateway_ips 2024-05-21T12:27:43.066 INFO:tasks.workunit.client.3.smithi137.stderr:+ IFS=, 2024-05-21T12:27:43.066 INFO:tasks.workunit.client.3.smithi137.stderr:+ read -ra gateway_names 2024-05-21T12:27:43.067 INFO:tasks.workunit.client.3.smithi137.stderr:++ echo 172.21.15.45,172.21.15.72,172.21.15.92 2024-05-21T12:27:43.067 INFO:tasks.workunit.client.3.smithi137.stderr:++ tr -cd , 2024-05-21T12:27:43.067 INFO:tasks.workunit.client.3.smithi137.stderr:++ wc -c 2024-05-21T12:27:43.076 INFO:tasks.workunit.client.3.smithi137.stderr:+ gateways_count=3 2024-05-21T12:27:43.076 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 3 2024-05-21T12:27:43.083 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:27:43.084 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode1 2024-05-21T12:27:43.084 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 subsystem add --subsystem nqn.2016-06.io.spdk:cnode1 2024-05-21T12:27:43.155 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:43Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:43.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:43 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:27:43.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:43 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:27:43.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:43 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:43.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:43 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:27:43.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:43 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:27:43.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:43 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:43.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:27:43.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:27:43.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:27:43.855 INFO:tasks.workunit.client.3.smithi137.stdout:Adding subsystem nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:27:44.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:43] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:27:44.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:43] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:27:44.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:43] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: , client address: IPv4 172.21.15.137:50780 2024-05-21T12:27:44.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:43] INFO grpc.py:541: No serial number specified for nqn.2016-06.io.spdk:cnode1, will use Ceph40086972576734 2024-05-21T12:27:44.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:44 smithi072 ceph-mon[33912]: pgmap v354: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:44.709 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:27:44.709 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode2 2024-05-21T12:27:44.710 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 subsystem add --subsystem nqn.2016-06.io.spdk:cnode2 2024-05-21T12:27:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:44 smithi092 ceph-mon[33665]: pgmap v354: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:44.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:44 smithi045 ceph-mon[30165]: pgmap v354: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2 op/s 2024-05-21T12:27:45.354 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:45Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:45] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:27:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 1 2024-05-21T12:27:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: OPTIMIZED 2024-05-21T12:27:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 2 2024-05-21T12:27:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 3 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 4 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 5 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 6 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 7 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 8 2024-05-21T12:27:45.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 9 2024-05-21T12:27:45.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 10 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 11 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 12 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 13 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 14 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 15 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 16 2024-05-21T12:27:45.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:45.824 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:45.824 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: ], , client address: IPv4 172.21.15.45:44352 2024-05-21T12:27:46.087 INFO:tasks.workunit.client.3.smithi137.stdout:Adding subsystem nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:27:46.406 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:46] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: , client address: IPv4 172.21.15.137:42634 2024-05-21T12:27:46.406 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:46] INFO grpc.py:541: No serial number specified for nqn.2016-06.io.spdk:cnode2, will use Ceph77937083268835 2024-05-21T12:27:46.407 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:46] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:27:46.407 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:46] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:27:46.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-mon[33912]: pgmap v355: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:46] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 2 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:27:46.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 3 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 4 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 5 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 6 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 7 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 8 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 9 2024-05-21T12:27:46.668 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 10 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 11 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.669 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 12 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 13 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 14 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.670 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.671 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 15 2024-05-21T12:27:46.671 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.671 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.671 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:46.671 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 16 2024-05-21T12:27:46.671 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:46.671 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:46.671 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: ], , client address: IPv4 172.21.15.72:60624 2024-05-21T12:27:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-mon[33665]: pgmap v355: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:46] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 3 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 4 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 5 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 6 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 7 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 8 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 9 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 10 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 11 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 12 2024-05-21T12:27:46.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 13 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 14 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 15 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:46.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 16 2024-05-21T12:27:46.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:46.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:46.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:59256 2024-05-21T12:27:46.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:46 smithi045 ceph-mon[30165]: pgmap v355: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-05-21T12:27:46.919 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:27:46.919 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode3 2024-05-21T12:27:46.920 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 subsystem add --subsystem nqn.2016-06.io.spdk:cnode3 2024-05-21T12:27:47.265 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:47Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:47.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:47] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:27:47.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 1 2024-05-21T12:27:47.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: OPTIMIZED 2024-05-21T12:27:47.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 2 2024-05-21T12:27:47.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 3 2024-05-21T12:27:47.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 4 2024-05-21T12:27:47.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 5 2024-05-21T12:27:47.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 6 2024-05-21T12:27:47.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 7 2024-05-21T12:27:47.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 8 2024-05-21T12:27:47.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 9 2024-05-21T12:27:47.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.823 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.824 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 10 2024-05-21T12:27:47.824 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.825 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.825 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.825 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 11 2024-05-21T12:27:47.825 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.825 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.825 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.826 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 12 2024-05-21T12:27:47.826 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.826 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.826 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.826 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 13 2024-05-21T12:27:47.826 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.827 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.827 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.827 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 14 2024-05-21T12:27:47.827 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.827 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.827 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.827 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 15 2024-05-21T12:27:47.828 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.828 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.828 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:47.828 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 16 2024-05-21T12:27:47.828 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:47.828 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:47.829 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: ], , client address: IPv4 172.21.15.45:60094 2024-05-21T12:27:48.022 INFO:tasks.workunit.client.3.smithi137.stdout:Adding subsystem nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:27:48.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:48] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: , client address: IPv4 172.21.15.137:41656 2024-05-21T12:27:48.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:48] INFO grpc.py:541: No serial number specified for nqn.2016-06.io.spdk:cnode3, will use Ceph15464245260738 2024-05-21T12:27:48.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:48] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:27:48.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:48] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:27:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-mon[33665]: pgmap v356: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-21T12:27:48.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:48] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:27:48.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:27:48.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:27:48.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 3 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 4 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 5 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 6 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 7 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 8 2024-05-21T12:27:48.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 9 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 10 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 11 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 12 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 13 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 14 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 15 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 16 2024-05-21T12:27:48.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:48.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:48.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:59266 2024-05-21T12:27:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:48 smithi045 ceph-mon[30165]: pgmap v356: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-21T12:27:48.863 INFO:tasks.workunit.client.3.smithi137.stderr:+ list_subsystems 2024-05-21T12:27:48.863 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:27:48.863 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.45 2024-05-21T12:27:48.863 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 --format json subsystem list 2024-05-21T12:27:48.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-mon[33912]: pgmap v356: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:48] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 2 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 3 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 4 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 5 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 6 2024-05-21T12:27:48.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 7 2024-05-21T12:27:48.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 8 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 9 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 10 2024-05-21T12:27:48.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 11 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 12 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 13 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 14 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 15 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 16 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:48.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: ], , client address: IPv4 172.21.15.72:60636 2024-05-21T12:27:49.163 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:49Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:49] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:27:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 1 2024-05-21T12:27:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: OPTIMIZED 2024-05-21T12:27:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 2 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 3 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 4 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 5 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 6 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 7 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 8 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 9 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 10 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.820 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 11 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 12 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 13 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 14 2024-05-21T12:27:49.821 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 15 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: states { 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: grp_id: 16 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: state: INACCESSIBLE 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: } 2024-05-21T12:27:49.822 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: ], , client address: IPv4 172.21.15.45:60112 2024-05-21T12:27:49.886 INFO:tasks.workunit.client.3.smithi137.stdout:{ 2024-05-21T12:27:49.887 INFO:tasks.workunit.client.3.smithi137.stdout: "error_message": "Success", 2024-05-21T12:27:49.887 INFO:tasks.workunit.client.3.smithi137.stdout: "subsystems": [ 2024-05-21T12:27:49.887 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:49.887 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode1", 2024-05-21T12:27:49.887 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:49.888 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph40086972576734", 2024-05-21T12:27:49.888 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:49.888 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 1, 2024-05-21T12:27:49.888 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 2040, 2024-05-21T12:27:49.888 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:49.888 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:49.889 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:49.889 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:27:49.889 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:49.889 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode2", 2024-05-21T12:27:49.889 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:49.889 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph77937083268835", 2024-05-21T12:27:49.890 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:49.890 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 1, 2024-05-21T12:27:49.890 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 2040, 2024-05-21T12:27:49.890 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:49.890 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:49.890 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:49.891 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:27:49.891 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:49.891 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode3", 2024-05-21T12:27:49.891 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:49.891 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph15464245260738", 2024-05-21T12:27:49.891 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:49.892 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 1, 2024-05-21T12:27:49.892 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 2040, 2024-05-21T12:27:49.892 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:49.892 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:49.892 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:49.892 INFO:tasks.workunit.client.3.smithi137.stdout: } 2024-05-21T12:27:49.892 INFO:tasks.workunit.client.3.smithi137.stdout: ], 2024-05-21T12:27:49.893 INFO:tasks.workunit.client.3.smithi137.stdout: "status": 0 2024-05-21T12:27:49.893 INFO:tasks.workunit.client.3.smithi137.stdout:} 2024-05-21T12:27:50.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:49] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:44658 2024-05-21T12:27:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-mon[33665]: pgmap v357: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:50] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 3 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 4 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 5 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 6 2024-05-21T12:27:50.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 7 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 8 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 9 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 10 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 11 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 12 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 13 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 14 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 15 2024-05-21T12:27:50.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:27:50.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 16 2024-05-21T12:27:50.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:27:50.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:27:50.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:59288 2024-05-21T12:27:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:50 smithi045 ceph-mon[30165]: pgmap v357: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:27:50.819 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:27:50.819 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.72 2024-05-21T12:27:50.819 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.72 --server-port 5500 --format json subsystem list 2024-05-21T12:27:50.892 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:50Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-mon[33912]: pgmap v357: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:50] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 2 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 3 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 4 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 5 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 6 2024-05-21T12:27:50.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 7 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 8 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 9 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.918 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 10 2024-05-21T12:27:50.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 11 2024-05-21T12:27:50.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.919 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 12 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 13 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 14 2024-05-21T12:27:50.920 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 15 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 16 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:27:50.921 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: ], , client address: IPv4 172.21.15.72:60656 2024-05-21T12:27:51.607 INFO:tasks.workunit.client.3.smithi137.stdout:{ 2024-05-21T12:27:51.607 INFO:tasks.workunit.client.3.smithi137.stdout: "error_message": "Success", 2024-05-21T12:27:51.607 INFO:tasks.workunit.client.3.smithi137.stdout: "subsystems": [ 2024-05-21T12:27:51.607 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:51.607 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode1", 2024-05-21T12:27:51.607 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph40086972576734", 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 2041, 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 4080, 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode2", 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph77937083268835", 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:51.608 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 2041, 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 4080, 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode3", 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph15464245260738", 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 2041, 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 4080, 2024-05-21T12:27:51.609 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:51.610 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:51.610 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:51.610 INFO:tasks.workunit.client.3.smithi137.stdout: } 2024-05-21T12:27:51.610 INFO:tasks.workunit.client.3.smithi137.stdout: ], 2024-05-21T12:27:51.610 INFO:tasks.workunit.client.3.smithi137.stdout: "status": 0 2024-05-21T12:27:51.610 INFO:tasks.workunit.client.3.smithi137.stdout:} 2024-05-21T12:27:51.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:51 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:51] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:50704 2024-05-21T12:27:52.561 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:27:52.562 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.92 2024-05-21T12:27:52.562 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.92 --server-port 5500 --format json subsystem list 2024-05-21T12:27:52.639 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:52Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:52 smithi092 ceph-mon[33665]: pgmap v358: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:27:52.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:52 smithi045 ceph-mon[30165]: pgmap v358: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:27:52.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:52 smithi072 ceph-mon[33912]: pgmap v358: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:27:53.365 INFO:tasks.workunit.client.3.smithi137.stdout:{ 2024-05-21T12:27:53.365 INFO:tasks.workunit.client.3.smithi137.stdout: "error_message": "Success", 2024-05-21T12:27:53.365 INFO:tasks.workunit.client.3.smithi137.stdout: "subsystems": [ 2024-05-21T12:27:53.365 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:53.365 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode1", 2024-05-21T12:27:53.365 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:53.365 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph40086972576734", 2024-05-21T12:27:53.365 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 4081, 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 6120, 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode2", 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph77937083268835", 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 4081, 2024-05-21T12:27:53.366 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 6120, 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode3", 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph15464245260738", 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 4081, 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 6120, 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256, 2024-05-21T12:27:53.367 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 0 2024-05-21T12:27:53.368 INFO:tasks.workunit.client.3.smithi137.stdout: } 2024-05-21T12:27:53.368 INFO:tasks.workunit.client.3.smithi137.stdout: ], 2024-05-21T12:27:53.368 INFO:tasks.workunit.client.3.smithi137.stdout: "status": 0 2024-05-21T12:27:53.368 INFO:tasks.workunit.client.3.smithi137.stdout:} 2024-05-21T12:27:53.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:53] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:57886 2024-05-21T12:27:54.273 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:27:54.273 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.45 2024-05-21T12:27:54.273 INFO:tasks.workunit.client.3.smithi137.stderr:+ name=smithi045 2024-05-21T12:27:54.274 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 3 2024-05-21T12:27:54.275 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:27:54.275 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode1 2024-05-21T12:27:54.275 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.45 and name smithi045' 2024-05-21T12:27:54.275 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.45 and name smithi045 2024-05-21T12:27:54.275 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode1 --host-name smithi045 --traddr 172.21.15.45 --trsvcid 4420 2024-05-21T12:27:54.359 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:54Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:54 smithi092 ceph-mon[33665]: pgmap v359: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-21T12:27:54.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:54 smithi045 ceph-mon[30165]: pgmap v359: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-21T12:27:54.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:54 smithi072 ceph-mon[33912]: pgmap v359: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 5.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-21T12:27:55.110 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode1 listener at 172.21.15.45:4420: Successful 2024-05-21T12:27:55.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:55 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:55] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: None 2024-05-21T12:27:55.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:55] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: None 2024-05-21T12:27:55.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:55] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: , client address: IPv4 172.21.15.137:57366 2024-05-21T12:27:55.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:27:55.085348] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.45 port 4420 *** 2024-05-21T12:27:56.017 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:27:56.017 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode2 2024-05-21T12:27:56.017 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.45 and name smithi045 2024-05-21T12:27:56.018 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.45 and name smithi045' 2024-05-21T12:27:56.018 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode2 --host-name smithi045 --traddr 172.21.15.45 --trsvcid 4420 2024-05-21T12:27:56.092 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:56Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:56 smithi092 ceph-mon[33665]: pgmap v360: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-21T12:27:56.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:56 smithi045 ceph-mon[30165]: pgmap v360: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-21T12:27:56.886 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:56 smithi072 ceph-mon[33912]: pgmap v360: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-21T12:27:56.887 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode2 listener at 172.21.15.45:4420: Successful 2024-05-21T12:27:57.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:56 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:56] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: None 2024-05-21T12:27:57.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:56] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: None 2024-05-21T12:27:57.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:56] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: , client address: IPv4 172.21.15.137:60450 2024-05-21T12:27:57.776 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.45 and name smithi045 2024-05-21T12:27:57.776 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:27:57.776 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode3 2024-05-21T12:27:57.777 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.45 and name smithi045' 2024-05-21T12:27:57.777 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode3 --host-name smithi045 --traddr 172.21.15.45 --trsvcid 4420 2024-05-21T12:27:57.862 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:57Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:27:58.570 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode3 listener at 172.21.15.45:4420: Successful 2024-05-21T12:27:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:27:58 smithi092 ceph-mon[33665]: pgmap v361: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 6.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-21T12:27:58.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:27:58 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:27:58] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: None 2024-05-21T12:27:58.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:27:58 smithi045 ceph-mon[30165]: pgmap v361: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 6.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-21T12:27:58.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:27:58 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:27:58] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: , client address: IPv4 172.21.15.137:42418 2024-05-21T12:27:58.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:27:58 smithi072 ceph-mon[33912]: pgmap v361: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 6.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-21T12:27:58.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:27:58 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:27:58] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: None 2024-05-21T12:27:59.502 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:27:59.502 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.72 2024-05-21T12:27:59.502 INFO:tasks.workunit.client.3.smithi137.stderr:+ name=smithi072 2024-05-21T12:27:59.503 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 3 2024-05-21T12:27:59.503 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:27:59.503 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode1 2024-05-21T12:27:59.504 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.72 and name smithi072 2024-05-21T12:27:59.504 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.72 and name smithi072' 2024-05-21T12:27:59.504 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.72 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode1 --host-name smithi072 --traddr 172.21.15.72 --trsvcid 4420 2024-05-21T12:27:59.584 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:27:59Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:00.279 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode1 listener at 172.21.15.72:4420: Successful 2024-05-21T12:28:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:00 smithi045 ceph-mon[30165]: pgmap v362: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:28:00.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:00 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:00] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: None 2024-05-21T12:28:00.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:00 smithi072 ceph-mon[33912]: pgmap v362: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:28:00.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:00 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:00] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: , client address: IPv4 172.21.15.137:38346 2024-05-21T12:28:00.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:00 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:00.254245] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.72 port 4420 *** 2024-05-21T12:28:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:00 smithi092 ceph-mon[33665]: pgmap v362: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:28:00.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:00] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: None 2024-05-21T12:28:01.178 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:01.179 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode2 2024-05-21T12:28:01.179 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.72 and name smithi072' 2024-05-21T12:28:01.179 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.72 and name smithi072 2024-05-21T12:28:01.179 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.72 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode2 --host-name smithi072 --traddr 172.21.15.72 --trsvcid 4420 2024-05-21T12:28:01.277 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:01Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:02.016 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode2 listener at 172.21.15.72:4420: Successful 2024-05-21T12:28:02.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:02 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:02] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: None 2024-05-21T12:28:02.418 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:01 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:01] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: , client address: IPv4 172.21.15.137:56500 2024-05-21T12:28:02.452 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:02 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:02] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: None 2024-05-21T12:28:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:02 smithi092 ceph-mon[33665]: pgmap v363: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:28:02.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:02 smithi045 ceph-mon[30165]: pgmap v363: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:28:02.880 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:02.881 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode3 2024-05-21T12:28:02.881 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.72 and name smithi072' 2024-05-21T12:28:02.881 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.72 and name smithi072 2024-05-21T12:28:02.881 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.72 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode3 --host-name smithi072 --traddr 172.21.15.72 --trsvcid 4420 2024-05-21T12:28:02.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:02 smithi072 ceph-mon[33912]: pgmap v363: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 341 B/s wr, 4 op/s 2024-05-21T12:28:02.959 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:02Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:03.715 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode3 listener at 172.21.15.72:4420: Successful 2024-05-21T12:28:04.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:03] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: None 2024-05-21T12:28:04.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:03 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:03] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: None 2024-05-21T12:28:04.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:03 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:03] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: , client address: IPv4 172.21.15.137:38848 2024-05-21T12:28:04.656 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:28:04.656 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.92 2024-05-21T12:28:04.657 INFO:tasks.workunit.client.3.smithi137.stderr:+ name=smithi092 2024-05-21T12:28:04.657 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 3 2024-05-21T12:28:04.657 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:04.657 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode1 2024-05-21T12:28:04.658 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.92 and name smithi092' 2024-05-21T12:28:04.658 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.92 and name smithi092 2024-05-21T12:28:04.658 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.92 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode1 --host-name smithi092 --traddr 172.21.15.92 --trsvcid 4420 2024-05-21T12:28:04.738 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:04Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:04 smithi092 ceph-mon[33665]: pgmap v364: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-21T12:28:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:04 smithi045 ceph-mon[30165]: pgmap v364: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-21T12:28:04.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:04 smithi072 ceph-mon[33912]: pgmap v364: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 8.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-21T12:28:05.465 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode1 listener at 172.21.15.92:4420: Successful 2024-05-21T12:28:05.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:05 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:05] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: , client address: IPv4 172.21.15.137:33796 2024-05-21T12:28:05.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:05 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:05.439877] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.92 port 4420 *** 2024-05-21T12:28:05.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:05 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:05] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: None 2024-05-21T12:28:05.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:05 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:05] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: None 2024-05-21T12:28:06.400 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:06.400 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode2 2024-05-21T12:28:06.400 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.92 and name smithi092' 2024-05-21T12:28:06.400 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.92 and name smithi092 2024-05-21T12:28:06.401 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.92 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode2 --host-name smithi092 --traddr 172.21.15.92 --trsvcid 4420 2024-05-21T12:28:06.486 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:06Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:06 smithi092 ceph-mon[33665]: pgmap v365: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 7.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-21T12:28:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:06 smithi045 ceph-mon[30165]: pgmap v365: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 7.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-21T12:28:06.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:06 smithi072 ceph-mon[33912]: pgmap v365: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 7.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-21T12:28:07.225 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode2 listener at 172.21.15.92:4420: Successful 2024-05-21T12:28:07.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:07 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:07] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: , client address: IPv4 172.21.15.137:54956 2024-05-21T12:28:07.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:07 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:07] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: None 2024-05-21T12:28:07.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:07 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:07] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: None 2024-05-21T12:28:08.126 INFO:tasks.workunit.client.3.smithi137.stderr:+ for j in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:08.127 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode3 2024-05-21T12:28:08.127 INFO:tasks.workunit.client.3.smithi137.stdout:Adding gateway listener with IP 172.21.15.92 and name smithi092 2024-05-21T12:28:08.127 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Adding gateway listener with IP 172.21.15.92 and name smithi092' 2024-05-21T12:28:08.127 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.92 --server-port 5500 listener add --subsystem nqn.2016-06.io.spdk:cnode3 --host-name smithi092 --traddr 172.21.15.92 --trsvcid 4420 2024-05-21T12:28:08.211 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:08Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:08 smithi092 ceph-mon[33665]: pgmap v366: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-21T12:28:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:08 smithi045 ceph-mon[30165]: pgmap v366: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-21T12:28:08.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:08 smithi072 ceph-mon[33912]: pgmap v366: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-21T12:28:08.965 INFO:tasks.workunit.client.3.smithi137.stdout:Adding nqn.2016-06.io.spdk:cnode3 listener at 172.21.15.92:4420: Successful 2024-05-21T12:28:09.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:08 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:08] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: , client address: IPv4 172.21.15.137:39022 2024-05-21T12:28:09.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:08] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: None 2024-05-21T12:28:09.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:08 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:08] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: None 2024-05-21T12:28:09.812 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 3 2024-05-21T12:28:09.813 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:09.813 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode1 2024-05-21T12:28:09.813 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 host add --subsystem nqn.2016-06.io.spdk:cnode1 --host '*' 2024-05-21T12:28:09.922 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:09Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:10.635 INFO:tasks.workunit.client.3.smithi137.stdout:Allowing open host access to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:10 smithi092 ceph-mon[33665]: pgmap v367: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:10.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:10 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:10] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: None 2024-05-21T12:28:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:10 smithi045 ceph-mon[30165]: pgmap v367: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:10.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:10 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:10] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: , client address: IPv4 172.21.15.137:43502 2024-05-21T12:28:10.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:10 smithi072 ceph-mon[33912]: pgmap v367: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:10.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:10 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:10] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: None 2024-05-21T12:28:11.521 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:11.521 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode2 2024-05-21T12:28:11.522 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 host add --subsystem nqn.2016-06.io.spdk:cnode2 --host '*' 2024-05-21T12:28:11.606 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:11Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:12.315 INFO:tasks.workunit.client.3.smithi137.stdout:Allowing open host access to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:28:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:12 smithi045 ceph-mon[30165]: pgmap v368: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:12.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:12] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: , client address: IPv4 172.21.15.137:49384 2024-05-21T12:28:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:12 smithi072 ceph-mon[33912]: pgmap v368: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:12.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:12 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:12] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: None 2024-05-21T12:28:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:12 smithi092 ceph-mon[33665]: pgmap v368: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:12.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:12 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:12] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: None 2024-05-21T12:28:13.182 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:13.182 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode3 2024-05-21T12:28:13.182 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 host add --subsystem nqn.2016-06.io.spdk:cnode3 --host '*' 2024-05-21T12:28:13.255 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:13Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:13.962 INFO:tasks.workunit.client.3.smithi137.stdout:Allowing open host access to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:28:14.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:13 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:13] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: None 2024-05-21T12:28:14.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:13] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: , client address: IPv4 172.21.15.137:32836 2024-05-21T12:28:14.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:13] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: None 2024-05-21T12:28:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:14 smithi092 ceph-mon[33665]: pgmap v369: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-21T12:28:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:14 smithi045 ceph-mon[30165]: pgmap v369: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-21T12:28:14.825 INFO:tasks.workunit.client.3.smithi137.stderr:+ image_index=1 2024-05-21T12:28:14.825 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 3 2024-05-21T12:28:14.826 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:14.826 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode1 2024-05-21T12:28:14.826 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 20 2024-05-21T12:28:14.827 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:14.827 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage1 2024-05-21T12:28:14.827 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage1 --load-balancing-group 2 2024-05-21T12:28:14.915 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:14Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:14.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:14 smithi072 ceph-mon[33912]: pgmap v369: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-21T12:28:15.647 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 1 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:15.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:15] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:15.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:15] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:15.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:15.658433] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.72:0/951708254, rc 0 2024-05-21T12:28:15.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:15.658529] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.72:0/951708254 2024-05-21T12:28:15.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:15] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.72:0/951708254' anagrp=2 2024-05-21T12:28:15.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:15] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:28:15.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:15.670921] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:28:15.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:15] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b 2024-05-21T12:28:16.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:15] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:16.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:15] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:16.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:15.660041] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.92:0/572653629, rc 0 2024-05-21T12:28:16.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:15.660143] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.92:0/572653629 2024-05-21T12:28:16.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:15] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.92:0/572653629' anagrp=2 2024-05-21T12:28:16.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:15] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:28:16.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:15.673252] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:28:16.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:15] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b 2024-05-21T12:28:16.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 2 context: , client address: IPv4 172.21.15.137:38482 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 18,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:38482 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:15.617826] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.45:0/3864753735, rc 0 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:15.617914] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.45:0/3864753735 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.45:0/3864753735' anagrp=2 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:15.631966] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:28:16.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:15 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:15] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using UUID 1ac8703e-ed04-493b-ba52-c73153370d7b , client address: IPv4 172.21.15.137:38482 2024-05-21T12:28:16.509 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:16.509 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:16.509 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage2 2024-05-21T12:28:16.509 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage2 --load-balancing-group 3 2024-05-21T12:28:16.592 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:16Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:16.755 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:16 smithi092 ceph-mon[33665]: pgmap v370: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:16.755 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:16 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/105113142' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:16 smithi045 ceph-mon[30165]: pgmap v370: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:16 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/105113142' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:16 smithi072 ceph-mon[33912]: pgmap v370: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-21T12:28:16.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:16 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/105113142' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:17.341 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 2 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:17.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:17] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 3 context: , client address: IPv4 172.21.15.137:38772 2024-05-21T12:28:17.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:17] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:17.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:17] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:38772 2024-05-21T12:28:17.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:17.311976] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.45:0/3455389857, rc 0 2024-05-21T12:28:17.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:17.312069] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.45:0/3455389857 2024-05-21T12:28:17.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:17] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.45:0/3455389857' anagrp=3 2024-05-21T12:28:17.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:17] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:28:17.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:17.326005] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:28:17.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:17 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:17] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b , client address: IPv4 172.21.15.137:38772 2024-05-21T12:28:17.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:17] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:17.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:17] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:17.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:17.350890] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.72:0/1296677301, rc 0 2024-05-21T12:28:17.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:17.351411] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.72:0/1296677301 2024-05-21T12:28:17.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:17] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.72:0/1296677301' anagrp=3 2024-05-21T12:28:17.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:17] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:28:17.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:17.364860] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:28:17.667 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:17] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b 2024-05-21T12:28:17.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:17 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:17] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:17.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:17 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:17] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:17.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:17 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:17.348843] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.92:0/3081013985, rc 0 2024-05-21T12:28:17.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:17 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:17.348922] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.92:0/3081013985 2024-05-21T12:28:17.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:17 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:17] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.92:0/3081013985' anagrp=3 2024-05-21T12:28:17.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:17 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:17] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:28:17.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:17 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:17.362425] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:28:17.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:17 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:17] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b 2024-05-21T12:28:18.187 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:18.187 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:18.187 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage3 2024-05-21T12:28:18.187 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage3 --load-balancing-group 1 2024-05-21T12:28:18.269 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:18Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:18 smithi092 ceph-mon[33665]: pgmap v371: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.6 KiB/s wr, 21 op/s 2024-05-21T12:28:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:18 smithi045 ceph-mon[30165]: pgmap v371: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.6 KiB/s wr, 21 op/s 2024-05-21T12:28:18.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:18 smithi072 ceph-mon[33912]: pgmap v371: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.6 KiB/s wr, 21 op/s 2024-05-21T12:28:19.054 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 3 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:19.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:19] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 1 context: , client address: IPv4 172.21.15.137:40560 2024-05-21T12:28:19.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:19] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:19.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:19] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:40560 2024-05-21T12:28:19.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:19.025878] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.45:0/3924692775, rc 0 2024-05-21T12:28:19.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:19.025961] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.45:0/3924692775 2024-05-21T12:28:19.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:19] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.45:0/3924692775' anagrp=1 2024-05-21T12:28:19.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:19] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:28:19.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:19.039451] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:28:19.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:19] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 , client address: IPv4 172.21.15.137:40560 2024-05-21T12:28:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:19] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:19] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:19.076794] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.72:0/3099482860, rc 0 2024-05-21T12:28:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:19.076900] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.72:0/3099482860 2024-05-21T12:28:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:19] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.72:0/3099482860' anagrp=1 2024-05-21T12:28:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:19] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:28:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:19.089575] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:28:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:19] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 2024-05-21T12:28:19.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:19] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:19.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:19] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:19.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:19.075872] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.92:0/4095921432, rc 0 2024-05-21T12:28:19.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:19.075985] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.92:0/4095921432 2024-05-21T12:28:19.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:19] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.92:0/4095921432' anagrp=1 2024-05-21T12:28:19.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:19] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:28:19.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:19.090459] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:28:19.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:19] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 2024-05-21T12:28:19.956 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:19.956 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:19.956 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage4 2024-05-21T12:28:19.957 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage4 --load-balancing-group 2 2024-05-21T12:28:20.050 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:20Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:20.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:20 smithi092 ceph-mon[33665]: pgmap v372: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.1 KiB/s wr, 16 op/s 2024-05-21T12:28:20.777 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:20 smithi045 ceph-mon[30165]: pgmap v372: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.1 KiB/s wr, 16 op/s 2024-05-21T12:28:20.803 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:20 smithi072 ceph-mon[33912]: pgmap v372: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.1 KiB/s wr, 16 op/s 2024-05-21T12:28:20.804 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 4 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:21.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:20] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 2 context: , client address: IPv4 172.21.15.137:57958 2024-05-21T12:28:21.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:20] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:21.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:20] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:57958 2024-05-21T12:28:21.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:20] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:28:21.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:20.790002] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:28:21.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:20 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:20] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e , client address: IPv4 172.21.15.137:57958 2024-05-21T12:28:21.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:20 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:20] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:21.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:20 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:20] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:21.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:20 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:20] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:28:21.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:20 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:20.813622] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:28:21.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:20 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:20] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e 2024-05-21T12:28:21.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:20 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:20] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:21.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:20 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:20] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:21.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:20 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:20] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:28:21.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:20 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:20.815068] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:28:21.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:20 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:20] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e 2024-05-21T12:28:21.651 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:21.651 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:21.651 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage5 2024-05-21T12:28:21.651 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage5 --load-balancing-group 3 2024-05-21T12:28:21.729 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:21Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:22.447 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 5 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:22 smithi092 ceph-mon[33665]: pgmap v373: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 1.2 KiB/s wr, 29 op/s 2024-05-21T12:28:22.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:22 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:22] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:22.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:22 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:22] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:22.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:22 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:22] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:28:22.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:22 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:22.459965] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:28:22.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:22 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:22] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 2024-05-21T12:28:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:22 smithi045 ceph-mon[30165]: pgmap v373: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 1.2 KiB/s wr, 29 op/s 2024-05-21T12:28:22.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:22] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 3 context: , client address: IPv4 172.21.15.137:39804 2024-05-21T12:28:22.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:22] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:22.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:22] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:39804 2024-05-21T12:28:22.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:22] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:28:22.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:22.434147] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:28:22.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:22] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 , client address: IPv4 172.21.15.137:39804 2024-05-21T12:28:22.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:22 smithi072 ceph-mon[33912]: pgmap v373: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 1.2 KiB/s wr, 29 op/s 2024-05-21T12:28:22.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:22 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:22] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:22.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:22 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:22] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:22.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:22 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:22] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:28:22.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:22 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:22.457175] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:28:22.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:22 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:22] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 2024-05-21T12:28:23.345 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:23.345 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:23.345 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage6 2024-05-21T12:28:23.345 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage6 --load-balancing-group 1 2024-05-21T12:28:23.428 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:23Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:24.149 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 6 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:24.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:24] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:24.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:24] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:24.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:24] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:28:24.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:24.158307] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:28:24.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:24] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 2024-05-21T12:28:24.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:24] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:24.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:24] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:24.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:24] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:28:24.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:24.163468] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:28:24.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:24] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 2024-05-21T12:28:24.510 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:24] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 1 context: , client address: IPv4 172.21.15.137:55928 2024-05-21T12:28:24.510 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:24] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:24.510 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:24] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:55928 2024-05-21T12:28:24.510 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:24] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:28:24.510 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:24.134188] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:28:24.510 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:24] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using UUID e349eac9-5fb1-4829-8523-49a423678213 , client address: IPv4 172.21.15.137:55928 2024-05-21T12:28:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:24 smithi045 ceph-mon[30165]: pgmap v374: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 1.8 KiB/s wr, 42 op/s 2024-05-21T12:28:24.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:24 smithi072 ceph-mon[33912]: pgmap v374: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 1.8 KiB/s wr, 42 op/s 2024-05-21T12:28:25.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:24 smithi092 ceph-mon[33665]: pgmap v374: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 1.8 KiB/s wr, 42 op/s 2024-05-21T12:28:25.033 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:25.033 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:25.034 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage7 2024-05-21T12:28:25.034 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage7 --load-balancing-group 2 2024-05-21T12:28:25.115 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:25Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:25.836 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 7 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:25] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 2 context: , client address: IPv4 172.21.15.137:42434 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:25] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:25] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 28,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:25] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:25] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:25] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:42434 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:25] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:25.822932] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:28:26.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:25 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:25] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 , client address: IPv4 172.21.15.137:42434 2024-05-21T12:28:26.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:25 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:25] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:26.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:25 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:25] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:26.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:25 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:25] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:28:26.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:25 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:25.846705] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:28:26.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:25 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:25] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 2024-05-21T12:28:26.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:25 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:25] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:26.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:25 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:25] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:26.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:25 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:25] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:28:26.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:25 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:25.848610] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:28:26.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:25 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:25] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 2024-05-21T12:28:26.695 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:26.695 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:26.695 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage8 2024-05-21T12:28:26.695 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage8 --load-balancing-group 3 2024-05-21T12:28:26.782 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:26Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:26 smithi045 ceph-mon[30165]: pgmap v375: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-05-21T12:28:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:26 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1836350123' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:26.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:26 smithi072 ceph-mon[33912]: pgmap v375: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-05-21T12:28:26.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:26 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1836350123' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:27.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:26 smithi092 ceph-mon[33665]: pgmap v375: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-05-21T12:28:27.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:26 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1836350123' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:27.497 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 8 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:27.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:27 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:27] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:27.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:27 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:27] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:27.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:27 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:27] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:28:27.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:27 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:27.509638] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:28:27.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:27 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:27] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af 2024-05-21T12:28:27.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:27 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:27] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 3 context: , client address: IPv4 172.21.15.137:46492 2024-05-21T12:28:27.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:27 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:27] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:27.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:27 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:27] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:46492 2024-05-21T12:28:27.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:27 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:27] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:28:27.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:27 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:27.483364] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:28:27.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:27 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:27] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using UUID b87a5ac5-b293-4485-b785-2a12b9a685af , client address: IPv4 172.21.15.137:46492 2024-05-21T12:28:27.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:27 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:27] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:27.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:27 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:27] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:27.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:27 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:27] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:28:27.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:27 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:27.507394] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:28:27.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:27 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:27] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af 2024-05-21T12:28:28.364 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:28.364 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:28.364 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage9 2024-05-21T12:28:28.364 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage9 --load-balancing-group 1 2024-05-21T12:28:28.438 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:28Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:28 smithi045 ceph-mon[30165]: pgmap v376: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 2.0 KiB/s wr, 57 op/s 2024-05-21T12:28:28.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:28 smithi072 ceph-mon[33912]: pgmap v376: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 2.0 KiB/s wr, 57 op/s 2024-05-21T12:28:29.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:28 smithi092 ceph-mon[33665]: pgmap v376: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 2.0 KiB/s wr, 57 op/s 2024-05-21T12:28:29.130 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 9 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:29.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:29] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:29.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:29] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:29.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:29] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:28:29.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:29.139832] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:28:29.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:29] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 2024-05-21T12:28:29.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:29] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:29.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:29] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:29.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:29] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:28:29.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:29.137573] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:28:29.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:29] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 2024-05-21T12:28:29.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:29] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 1 context: , client address: IPv4 172.21.15.137:39464 2024-05-21T12:28:29.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:29] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:29.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:29] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:39464 2024-05-21T12:28:29.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:29] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:28:29.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:29.116334] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:28:29.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:29] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 , client address: IPv4 172.21.15.137:39464 2024-05-21T12:28:29.967 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:29.967 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:29.967 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage10 2024-05-21T12:28:29.967 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage10 --load-balancing-group 2 2024-05-21T12:28:30.051 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:30Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:30.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:30 smithi045 ceph-mon[30165]: pgmap v377: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-05-21T12:28:30.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:30 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:30] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 2 context: , client address: IPv4 172.21.15.137:33992 2024-05-21T12:28:30.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:30 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:30] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:30.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:30 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:30] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:33992 2024-05-21T12:28:30.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:30 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:30] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:28:30.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:30 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:30.784419] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:28:30.798 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:30 smithi092 ceph-mon[33665]: pgmap v377: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-05-21T12:28:30.799 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:30 smithi072 ceph-mon[33912]: pgmap v377: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-05-21T12:28:30.799 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 10 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:31.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:30 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:30] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using UUID cba00c90-3501-4d43-b5c2-017aac10a02d , client address: IPv4 172.21.15.137:33992 2024-05-21T12:28:31.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:30 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:30] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:31.167 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:30 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:30] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:31.167 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:30 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:30] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:28:31.167 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:30 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:30.808008] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:28:31.167 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:30 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:30] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d 2024-05-21T12:28:31.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:30 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:30] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:31.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:30 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:30] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:31.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:30 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:30] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:28:31.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:30 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:30.810835] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:28:31.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:30 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:30] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d 2024-05-21T12:28:31.659 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:31.659 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:31.659 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage11 2024-05-21T12:28:31.660 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage11 --load-balancing-group 3 2024-05-21T12:28:31.743 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:31Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:32.438 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 11 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:32 smithi092 ceph-mon[33665]: pgmap v378: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:28:32.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:32 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:32] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:32.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:32 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:32] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:32.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:32 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:32] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:28:32.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:32 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:32.446735] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:28:32.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:32 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:32] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 2024-05-21T12:28:32.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:32 smithi045 ceph-mon[30165]: pgmap v378: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:28:32.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:32 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:32] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 3 context: , client address: IPv4 172.21.15.137:44376 2024-05-21T12:28:32.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:32 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:32] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:32.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:32 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:32] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:44376 2024-05-21T12:28:32.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:32 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:32] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:28:32.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:32 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:32.424116] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:28:32.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:32 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:32] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 , client address: IPv4 172.21.15.137:44376 2024-05-21T12:28:32.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:32 smithi072 ceph-mon[33912]: pgmap v378: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:28:32.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:32 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:32] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:32.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:32 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:32] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:32.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:32 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:32] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:28:32.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:32 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:32.447900] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:28:32.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:32 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:32] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 2024-05-21T12:28:33.337 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:33.337 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:33.337 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage12 2024-05-21T12:28:33.337 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage12 --load-balancing-group 1 2024-05-21T12:28:33.432 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:33Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:34.162 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 12 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:34.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:33 smithi092 ceph-mon[33665]: pgmap v379: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 2.0 KiB/s wr, 59 op/s 2024-05-21T12:28:34.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:34] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:34.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:34] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:34.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:34] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:28:34.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:34.175278] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:28:34.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:34] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e 2024-05-21T12:28:34.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:33 smithi045 ceph-mon[30165]: pgmap v379: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 2.0 KiB/s wr, 59 op/s 2024-05-21T12:28:34.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:34 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:34] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 1 context: , client address: IPv4 172.21.15.137:55888 2024-05-21T12:28:34.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:34 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:34] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:34.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:34 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:34] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:55888 2024-05-21T12:28:34.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:34 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:34] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:28:34.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:34 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:34.147292] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:28:34.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:34 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:34] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e , client address: IPv4 172.21.15.137:55888 2024-05-21T12:28:34.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:33 smithi072 ceph-mon[33912]: pgmap v379: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 2.0 KiB/s wr, 59 op/s 2024-05-21T12:28:34.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:34 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:34] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:34.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:34 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:34] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:34.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:34 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:34] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:28:34.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:34 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:34.172507] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:28:34.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:34 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:34] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e 2024-05-21T12:28:35.020 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:35.020 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:35.020 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage13 2024-05-21T12:28:35.021 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage13 --load-balancing-group 2 2024-05-21T12:28:35.112 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:35Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:35.835 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 13 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:35] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 2 context: , client address: IPv4 172.21.15.137:39228 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:35] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:35] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 37,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:35] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:35] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:35] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:39228 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:35] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:35.821096] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:28:36.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:35] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using UUID 513f8027-44a0-43dc-82c0-6df74017b797 , client address: IPv4 172.21.15.137:39228 2024-05-21T12:28:36.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:35] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:36.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:35] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:36.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:35] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:28:36.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:35.843688] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:28:36.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:35] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 2024-05-21T12:28:36.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:35] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:36.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:35] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:36.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:35] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:28:36.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:35.847025] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:28:36.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:35] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 2024-05-21T12:28:36.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:36 smithi072 ceph-mon[33912]: pgmap v380: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:28:36.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:36 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3608073231' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:36 smithi092 ceph-mon[33665]: pgmap v380: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:28:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:36 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3608073231' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:36.754 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:36.754 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:36.755 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage14 2024-05-21T12:28:36.755 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage14 --load-balancing-group 3 2024-05-21T12:28:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:36 smithi045 ceph-mon[30165]: pgmap v380: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:28:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:36 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3608073231' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:36.841 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:36Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:37.612 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 14 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:37.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:37 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:37] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:37.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:37 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:37] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:37.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:37 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:37] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:28:37.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:37 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:37.624399] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:28:37.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:37 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:37] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 2024-05-21T12:28:38.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:37 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:37] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:38.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:37 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:37] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:38.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:37 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:37] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:28:38.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:37 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:37.622765] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:28:38.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:37 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:37] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 2024-05-21T12:28:38.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:37 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:37] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 3 context: , client address: IPv4 172.21.15.137:52412 2024-05-21T12:28:38.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:37 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:37] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:38.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:37 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:37] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:52412 2024-05-21T12:28:38.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:37 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:37] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:28:38.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:37 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:37.595586] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:28:38.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:37 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:37] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 , client address: IPv4 172.21.15.137:52412 2024-05-21T12:28:38.506 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:38.506 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:38.506 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage15 2024-05-21T12:28:38.506 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage15 --load-balancing-group 1 2024-05-21T12:28:38.581 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:38Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:38 smithi072 ceph-mon[33912]: pgmap v381: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:28:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:38 smithi092 ceph-mon[33665]: pgmap v381: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:28:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:38 smithi045 ceph-mon[30165]: pgmap v381: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:28:39.297 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 15 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:39.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:39] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 1 context: , client address: IPv4 172.21.15.137:42728 2024-05-21T12:28:39.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:39] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:39.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:39] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:42728 2024-05-21T12:28:39.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:39] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:28:39.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:39.281585] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:28:39.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:39] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using UUID 346f90e2-ed39-4090-8599-e1283c82d3ad , client address: IPv4 172.21.15.137:42728 2024-05-21T12:28:39.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:39 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:39] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:39.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:39 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:39] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:39.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:39 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:39] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:28:39.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:39 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:39.308835] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:28:39.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:39 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:39] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad 2024-05-21T12:28:39.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:39 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:39] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:39.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:39 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:39] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:39.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:39 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:39] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:28:39.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:39 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:39.306936] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:28:39.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:39 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:39] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad 2024-05-21T12:28:40.183 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:40.183 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:40.183 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage16 2024-05-21T12:28:40.183 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage16 --load-balancing-group 2 2024-05-21T12:28:40.270 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:40Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:40.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:40 smithi072 ceph-mon[33912]: pgmap v382: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:28:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:40 smithi092 ceph-mon[33665]: pgmap v382: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:28:40.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:40 smithi045 ceph-mon[30165]: pgmap v382: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:28:41.009 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 16 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:41.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:40] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 2 context: , client address: IPv4 172.21.15.137:56936 2024-05-21T12:28:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:40] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:40] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:56936 2024-05-21T12:28:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:28:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:40.995327] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:28:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:40] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 , client address: IPv4 172.21.15.137:56936 2024-05-21T12:28:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:41] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:41.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:41] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:41.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:28:41.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:41.020975] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:28:41.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:41] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 2024-05-21T12:28:41.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:41] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:41.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:41] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:41.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:28:41.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:41.019531] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:28:41.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:41] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 2024-05-21T12:28:41.875 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:41.875 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:41.875 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage17 2024-05-21T12:28:41.875 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage17 --load-balancing-group 3 2024-05-21T12:28:41.966 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:41Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:42.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:42 smithi072 ceph-mon[33912]: pgmap v383: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 1.7 KiB/s wr, 72 op/s 2024-05-21T12:28:42.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:42 smithi045 ceph-mon[30165]: pgmap v383: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 1.7 KiB/s wr, 72 op/s 2024-05-21T12:28:42.699 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 17 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:42.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:42 smithi092 ceph-mon[33665]: pgmap v383: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 1.7 KiB/s wr, 72 op/s 2024-05-21T12:28:43.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:42] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:43.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:42] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:43.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:28:43.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:42.714738] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:28:43.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:42] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 2024-05-21T12:28:43.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:42] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 3 context: , client address: IPv4 172.21.15.137:54226 2024-05-21T12:28:43.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:42] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:43.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:42] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:54226 2024-05-21T12:28:43.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:28:43.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:42.683464] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:28:43.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:42 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:42] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 , client address: IPv4 172.21.15.137:54226 2024-05-21T12:28:43.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:42] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:43.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:42] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:43.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:28:43.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:42.710814] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:28:43.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:42] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 2024-05-21T12:28:43.658 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:43.658 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:43.658 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage18 2024-05-21T12:28:43.659 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage18 --load-balancing-group 1 2024-05-21T12:28:43.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:43 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:28:43.750 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:43Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:43.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:43 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:28:43.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:43 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:28:44.498 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 18 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: pgmap v384: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:28:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:44] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:44.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:44] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:44.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:44] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:28:44.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:44.511044] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:28:44.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:44] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a 2024-05-21T12:28:44.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: pgmap v384: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:28:44.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:44] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 1 context: , client address: IPv4 172.21.15.137:38634 2024-05-21T12:28:44.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:44] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:44.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:44] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:38634 2024-05-21T12:28:44.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:44] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:28:44.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:44.484210] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:28:44.819 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:44 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:44] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a , client address: IPv4 172.21.15.137:38634 2024-05-21T12:28:44.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: pgmap v384: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:28:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:44.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:44] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:28:44.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:44] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:44.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:44] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:28:44.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:44.512060] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:28:44.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:44] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a 2024-05-21T12:28:45.426 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:45.426 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:45.426 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage19 2024-05-21T12:28:45.426 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage19 --load-balancing-group 2 2024-05-21T12:28:45.517 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:45Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:45.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:45 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-05-21T12:28:45.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:45 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:28:45.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:45 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:28:45.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:45 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:46.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:45 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-05-21T12:28:46.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:45 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:28:46.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:45 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:28:46.003 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:45 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:46.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:45 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-05-21T12:28:46.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:45 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:28:46.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:45 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:28:46.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:45 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:28:46.255 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 19 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:46] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 2 context: , client address: IPv4 172.21.15.137:35394 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:46] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:46] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 47,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:46] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:46] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:46] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:35394 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:46] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:46.240015] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:28:46.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:46] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b , client address: IPv4 172.21.15.137:35394 2024-05-21T12:28:46.655 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:46] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:46.655 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:46] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:46.655 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:46] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:28:46.655 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:46.267024] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:28:46.655 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:46] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b 2024-05-21T12:28:46.655 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:46] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:28:46.655 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:46] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:46.655 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:46] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:28:46.656 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:46.265268] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:28:46.656 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:46 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:46] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b 2024-05-21T12:28:46.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:46 smithi072 ceph-mon[33912]: pgmap v385: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:28:46.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:46 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/4156377034' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:47.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:46 smithi092 ceph-mon[33665]: pgmap v385: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:28:47.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:46 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/4156377034' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:47.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-mon[30165]: pgmap v385: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:28:47.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:46 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/4156377034' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:47.130 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:47.130 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:47.131 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage20 2024-05-21T12:28:47.131 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode1 --rbd-pool mypool --rbd-image myimage20 --load-balancing-group 3 2024-05-21T12:28:47.223 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:47Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:47.942 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 20 to nqn.2016-06.io.spdk:cnode1: Successful 2024-05-21T12:28:48.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:47] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:48.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:47] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:48.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:47] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:28:48.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:47.951167] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:28:48.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:47] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc 2024-05-21T12:28:48.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:47] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode1, ana group 3 context: , client address: IPv4 172.21.15.137:41884 2024-05-21T12:28:48.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:47] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:48.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:47] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:41884 2024-05-21T12:28:48.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:47] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:28:48.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:47.926267] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:28:48.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:47] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc , client address: IPv4 172.21.15.137:41884 2024-05-21T12:28:48.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:47] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:28:48.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:47] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:48.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:47] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:28:48.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:47.953343] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:28:48.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:47] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc 2024-05-21T12:28:48.823 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:48.824 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:28:48.824 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode2 2024-05-21T12:28:48.824 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 20 2024-05-21T12:28:48.825 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:48.825 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage21 2024-05-21T12:28:48.825 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage21 --load-balancing-group 1 2024-05-21T12:28:48.903 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:48Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:48.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:48 smithi072 ceph-mon[33912]: pgmap v386: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 2.2 KiB/s wr, 73 op/s 2024-05-21T12:28:49.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:48 smithi092 ceph-mon[33665]: pgmap v386: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 2.2 KiB/s wr, 73 op/s 2024-05-21T12:28:49.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:48 smithi045 ceph-mon[30165]: pgmap v386: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 2.2 KiB/s wr, 73 op/s 2024-05-21T12:28:49.581 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 1 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:28:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:49] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 1 context: , client address: IPv4 172.21.15.137:58048 2024-05-21T12:28:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:49] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:49] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:58048 2024-05-21T12:28:49.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:49] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:28:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:49.567294] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:28:49.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:49] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using UUID 88372a05-54d9-4c17-af42-2b9c02991736 , client address: IPv4 172.21.15.137:58048 2024-05-21T12:28:49.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:49 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:49] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:28:49.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:49 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:49] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:49.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:49 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:49] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:28:49.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:49 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:49.593754] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:28:49.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:49 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:49] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 2024-05-21T12:28:50.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:49] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:28:50.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:49] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:50.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:49] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:28:50.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:49.590721] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:28:50.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:49] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 2024-05-21T12:28:50.460 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:50.460 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:50.460 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage22 2024-05-21T12:28:50.460 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage22 --load-balancing-group 2 2024-05-21T12:28:50.538 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:50Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:50.920 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:50 smithi092 ceph-mon[33665]: pgmap v387: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:28:51.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:50 smithi045 ceph-mon[30165]: pgmap v387: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:28:51.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:50 smithi072 ceph-mon[33912]: pgmap v387: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:28:51.217 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 2 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:28:51.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:51] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:28:51.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:51] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:51.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:51] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:28:51.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:51.226439] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:28:51.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:51] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed 2024-05-21T12:28:51.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:51] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 2 context: , client address: IPv4 172.21.15.137:56000 2024-05-21T12:28:51.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:51] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:51.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:51] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:56000 2024-05-21T12:28:51.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:51] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:28:51.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:51.202605] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:28:51.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:51] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed , client address: IPv4 172.21.15.137:56000 2024-05-21T12:28:51.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:51 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:51] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:28:51.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:51 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:51] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:51.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:51 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:51] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:28:51.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:51 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:51.225611] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:28:51.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:51 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:51] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed 2024-05-21T12:28:52.071 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:52.071 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:52.071 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage23 2024-05-21T12:28:52.071 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage23 --load-balancing-group 3 2024-05-21T12:28:52.146 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:52Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:52.869 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 3 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:28:53.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:52 smithi092 ceph-mon[33665]: pgmap v388: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:28:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:52] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:28:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:52] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:28:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:52.879832] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:28:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:52] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc 2024-05-21T12:28:53.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:52 smithi045 ceph-mon[30165]: pgmap v388: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:28:53.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:52] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 3 context: , client address: IPv4 172.21.15.137:48496 2024-05-21T12:28:53.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:52] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:53.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:52] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:48496 2024-05-21T12:28:53.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:28:53.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:52.855709] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:28:53.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:52] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc , client address: IPv4 172.21.15.137:48496 2024-05-21T12:28:53.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:52 smithi072 ceph-mon[33912]: pgmap v388: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:28:53.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:52] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:28:53.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:52] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:53.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:28:53.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:52.882229] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:28:53.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:52] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc 2024-05-21T12:28:53.731 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:53.731 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:53.731 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage24 2024-05-21T12:28:53.731 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage24 --load-balancing-group 1 2024-05-21T12:28:53.814 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:53Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:54.539 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 4 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:28:54.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:54 smithi045 ceph-mon[30165]: pgmap v389: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:28:54.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:54 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:54] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 1 context: , client address: IPv4 172.21.15.137:58040 2024-05-21T12:28:54.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:54 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:54] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:54.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:54 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:54] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:58040 2024-05-21T12:28:54.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:54 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:28:54.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:54 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:54.524231] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:28:54.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:54 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:54] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b , client address: IPv4 172.21.15.137:58040 2024-05-21T12:28:54.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:54 smithi072 ceph-mon[33912]: pgmap v389: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:28:54.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:54] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:28:54.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:54] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:54.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:28:54.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:54.550312] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:28:54.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:54] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b 2024-05-21T12:28:55.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:54 smithi092 ceph-mon[33665]: pgmap v389: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:28:55.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:54] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:28:55.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:54] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:55.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:28:55.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:54.548575] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:28:55.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:54] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b 2024-05-21T12:28:55.407 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:55.407 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:55.407 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage25 2024-05-21T12:28:55.407 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage25 --load-balancing-group 2 2024-05-21T12:28:55.479 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:55Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:56.164 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 5 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:28:56.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:56 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:56] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:28:56.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:56 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:56] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:56.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:56 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:56] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:28:56.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:56 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:56.175690] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:28:56.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:56 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:56] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 2024-05-21T12:28:56.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:56] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:28:56.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:56] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:56.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:56] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:28:56.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:56.172747] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:28:56.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:56] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 2024-05-21T12:28:56.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:56] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 2 context: , client address: IPv4 172.21.15.137:39810 2024-05-21T12:28:56.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:56] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:56.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:56] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:39810 2024-05-21T12:28:56.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:56] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:28:56.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:56.149536] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:28:56.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:56 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:56] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using UUID 5aaed34a-1696-499e-b0b0-fad88d740507 , client address: IPv4 172.21.15.137:39810 2024-05-21T12:28:57.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:56 smithi092 ceph-mon[33665]: pgmap v390: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:28:57.041 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:57.042 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:57.042 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage26 2024-05-21T12:28:57.042 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage26 --load-balancing-group 3 2024-05-21T12:28:57.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:56 smithi045 ceph-mon[30165]: pgmap v390: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:28:57.120 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:57Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:57.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:56 smithi072 ceph-mon[33912]: pgmap v390: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:28:57.859 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 6 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:28:58.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:57 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:57] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:28:58.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:57 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:57] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:58.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:57 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:57] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:28:58.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:57 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:57.868277] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:28:58.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:57 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:57] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 2024-05-21T12:28:58.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:57] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:28:58.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:57] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:58.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:57] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:28:58.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:57.868759] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:28:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:57] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 2024-05-21T12:28:58.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:57] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 3 context: , client address: IPv4 172.21.15.137:57130 2024-05-21T12:28:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:57] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:28:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:57] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 58,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:28:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:57] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:28:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:57] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:28:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:57] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:57130 2024-05-21T12:28:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:57] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:28:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:57.842959] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:28:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:57] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using UUID c21863b3-ea38-4777-afc0-6a7011958143 , client address: IPv4 172.21.15.137:57130 2024-05-21T12:28:58.709 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:28:58.709 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:28:58.709 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage27 2024-05-21T12:28:58.709 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage27 --load-balancing-group 1 2024-05-21T12:28:58.787 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:28:58Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:28:59.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:58 smithi092 ceph-mon[33665]: pgmap v391: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:28:59.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:28:58 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/328594467' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:59.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:58 smithi045 ceph-mon[30165]: pgmap v391: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:28:59.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:28:58 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/328594467' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:59.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:58 smithi072 ceph-mon[33912]: pgmap v391: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:28:59.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:28:58 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/328594467' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:28:59.526 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 7 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:28:59.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:59] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 1 context: , client address: IPv4 172.21.15.137:53026 2024-05-21T12:28:59.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:59] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:28:59.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:59] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:53026 2024-05-21T12:28:59.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:59] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:28:59.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:28:59.511950] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:28:59.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:28:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:28:59] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 , client address: IPv4 172.21.15.137:53026 2024-05-21T12:28:59.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:59 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:59] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:28:59.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:59 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:59] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:28:59.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:59 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:59] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:28:59.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:59 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:28:59.536654] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:28:59.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:28:59 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:28:59] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 2024-05-21T12:29:00.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:59] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:00.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:59] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:00.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:59] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:29:00.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:28:59.537363] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:29:00.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:28:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:28:59] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 2024-05-21T12:29:00.470 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:00.470 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:00.470 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage28 2024-05-21T12:29:00.470 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage28 --load-balancing-group 2 2024-05-21T12:29:00.556 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:00Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:01.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:00 smithi092 ceph-mon[33665]: pgmap v392: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 1.6 KiB/s wr, 58 op/s 2024-05-21T12:29:01.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:00 smithi045 ceph-mon[30165]: pgmap v392: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 1.6 KiB/s wr, 58 op/s 2024-05-21T12:29:01.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:00 smithi072 ceph-mon[33912]: pgmap v392: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 1.6 KiB/s wr, 58 op/s 2024-05-21T12:29:01.304 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 8 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:01.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:01 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:01] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 2 context: , client address: IPv4 172.21.15.137:53820 2024-05-21T12:29:01.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:01 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:01] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:01.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:01 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:01] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:53820 2024-05-21T12:29:01.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:01 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:01] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:29:01.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:01 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:01.288575] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:29:01.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:01 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:01] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 , client address: IPv4 172.21.15.137:53820 2024-05-21T12:29:01.610 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:01] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:01.611 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:01] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:01.611 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:01] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:29:01.611 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:01.313940] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:29:01.611 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:01] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 2024-05-21T12:29:01.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:01 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:01] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:01.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:01 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:01] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:01.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:01 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:01] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:29:01.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:01 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:01.313490] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:29:01.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:01 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:01] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 2024-05-21T12:29:02.221 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:02.221 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:02.221 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage29 2024-05-21T12:29:02.221 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage29 --load-balancing-group 3 2024-05-21T12:29:02.302 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:02Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:03.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:02 smithi045 ceph-mon[30165]: pgmap v393: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:03.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:02 smithi092 ceph-mon[33665]: pgmap v393: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:03.028 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:02 smithi072 ceph-mon[33912]: pgmap v393: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:03.029 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 9 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:03.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:03 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:03] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 3 context: , client address: IPv4 172.21.15.137:60698 2024-05-21T12:29:03.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:03 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:03] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:03.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:03 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:03] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:60698 2024-05-21T12:29:03.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:03 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:03] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:29:03.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:03 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:03.012256] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:29:03.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:03 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:03] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea , client address: IPv4 172.21.15.137:60698 2024-05-21T12:29:03.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:03 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:03] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:29:03.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:03 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:03] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:03.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:03 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:03] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:29:03.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:03 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:03.039260] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:29:03.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:03 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:03] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea 2024-05-21T12:29:03.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:03] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:29:03.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:03] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:03.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:03] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:29:03.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:03.040049] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:29:03.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:03] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea 2024-05-21T12:29:03.906 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:03.906 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:03.906 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage30 2024-05-21T12:29:03.906 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage30 --load-balancing-group 1 2024-05-21T12:29:03.978 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:03Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:04.757 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 10 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:05.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:04 smithi092 ceph-mon[33665]: pgmap v394: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:05.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:04 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:04] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:05.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:04 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:04] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:05.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:04 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:04] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:29:05.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:04 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:04.770119] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:29:05.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:04 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:04] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 2024-05-21T12:29:05.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:04 smithi045 ceph-mon[30165]: pgmap v394: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:05.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:04] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 1 context: , client address: IPv4 172.21.15.137:44362 2024-05-21T12:29:05.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:04] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:05.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:04] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:44362 2024-05-21T12:29:05.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:04] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:29:05.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:04.740820] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:29:05.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:04] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 , client address: IPv4 172.21.15.137:44362 2024-05-21T12:29:05.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:04 smithi072 ceph-mon[33912]: pgmap v394: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:05.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:04 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:04] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:05.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:04 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:04] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:05.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:04 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:04] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:29:05.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:04 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:04.768894] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:29:05.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:04 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:04] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 2024-05-21T12:29:05.617 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:05.617 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:05.617 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage31 2024-05-21T12:29:05.617 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage31 --load-balancing-group 2 2024-05-21T12:29:05.694 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:05Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:06.407 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 11 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:06.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:06 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:06] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:06.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:06 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:06] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:06.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:06 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:06] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:29:06.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:06 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:06.416674] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:29:06.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:06 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:06] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 2024-05-21T12:29:06.729 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:06 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:06] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 2 context: , client address: IPv4 172.21.15.137:54350 2024-05-21T12:29:06.729 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:06 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:06] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:06.729 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:06 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:06] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:54350 2024-05-21T12:29:06.729 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:06 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:06] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:29:06.729 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:06 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:06.391074] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:29:06.729 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:06 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:06] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 , client address: IPv4 172.21.15.137:54350 2024-05-21T12:29:06.730 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:06 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:06] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:06.730 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:06 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:06] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:06.730 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:06 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:06] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:29:06.730 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:06 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:06.418505] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:29:06.730 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:06 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:06] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 2024-05-21T12:29:07.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:06 smithi092 ceph-mon[33665]: pgmap v395: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:07.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:06 smithi045 ceph-mon[30165]: pgmap v395: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:07.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:06 smithi072 ceph-mon[33912]: pgmap v395: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:07.268 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:07.268 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:07.268 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage32 2024-05-21T12:29:07.268 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage32 --load-balancing-group 3 2024-05-21T12:29:07.358 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:07Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:08.083 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 12 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:08.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:08] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 3 context: , client address: IPv4 172.21.15.137:55832 2024-05-21T12:29:08.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:08] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:29:08.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:08] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 68,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:29:08.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:08] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:29:08.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:08] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:29:08.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:08] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:55832 2024-05-21T12:29:08.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:08] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:29:08.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:08.068093] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:29:08.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:08] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 , client address: IPv4 172.21.15.137:55832 2024-05-21T12:29:08.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:08 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:08] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:29:08.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:08 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:08] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:08.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:08 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:08] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:29:08.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:08 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:08.092862] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:29:08.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:08 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:08] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 2024-05-21T12:29:08.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:08 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:08] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:29:08.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:08 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:08] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:08.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:08 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:08] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:29:08.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:08 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:08.094640] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:29:08.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:08 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:08] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 2024-05-21T12:29:08.951 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:08.951 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:08.951 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage33 2024-05-21T12:29:08.951 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage33 --load-balancing-group 1 2024-05-21T12:29:09.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:08 smithi092 ceph-mon[33665]: pgmap v396: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:09.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:08 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2481631814' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:09.040 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:09Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:09.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-mon[30165]: pgmap v396: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:09.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:08 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2481631814' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:09.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:08 smithi072 ceph-mon[33912]: pgmap v396: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:09.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:08 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2481631814' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:09.769 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 13 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:10.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:09 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:09] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 1 context: , client address: IPv4 172.21.15.137:42030 2024-05-21T12:29:10.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:09 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:09] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:10.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:09 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:09] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:42030 2024-05-21T12:29:10.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:09 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:09] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:29:10.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:09 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:09.753674] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:29:10.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:09 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:09] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 , client address: IPv4 172.21.15.137:42030 2024-05-21T12:29:10.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:09 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:09] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:10.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:09 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:09] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:10.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:09 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:09] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:29:10.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:09 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:09.780885] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:29:10.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:09 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:09] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 2024-05-21T12:29:10.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:09 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:09] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:10.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:09 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:09] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:10.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:09 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:09] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:29:10.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:09 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:09.778338] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:29:10.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:09 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:09] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 2024-05-21T12:29:10.677 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:10.677 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:10.677 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage34 2024-05-21T12:29:10.678 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage34 --load-balancing-group 2 2024-05-21T12:29:10.763 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:10Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:11.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:10 smithi092 ceph-mon[33665]: pgmap v397: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:11.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:10 smithi045 ceph-mon[30165]: pgmap v397: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:11.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:10 smithi072 ceph-mon[33912]: pgmap v397: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:11.508 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 14 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:11.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:11 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:11] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 2 context: , client address: IPv4 172.21.15.137:46476 2024-05-21T12:29:11.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:11 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:11] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:11.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:11 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:11] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:46476 2024-05-21T12:29:11.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:11 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:11] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:29:11.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:11 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:11.493726] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:29:11.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:11 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:11] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a , client address: IPv4 172.21.15.137:46476 2024-05-21T12:29:11.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:11 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:11] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:11.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:11 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:11] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:11.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:11 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:11] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:29:11.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:11 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:11.517158] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:29:11.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:11 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:11] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a 2024-05-21T12:29:12.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:11 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:11] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:12.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:11 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:11] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:12.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:11 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:11] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:29:12.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:11 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:11.518651] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:29:12.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:11 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:11] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a 2024-05-21T12:29:12.346 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:12.346 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:12.346 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage35 2024-05-21T12:29:12.346 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage35 --load-balancing-group 3 2024-05-21T12:29:12.429 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:12Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:13.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:12 smithi092 ceph-mon[33665]: pgmap v398: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:13.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:12 smithi045 ceph-mon[30165]: pgmap v398: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:13.120 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 15 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:13.120 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:12 smithi072 ceph-mon[33912]: pgmap v398: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:13.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:13] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:29:13.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:13] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:13.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:29:13.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:13.130784] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:29:13.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:13] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 2024-05-21T12:29:13.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:13 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:13] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:29:13.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:13 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:13] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:13.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:13 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:29:13.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:13 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:13.133774] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:29:13.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:13 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:13] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 2024-05-21T12:29:13.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:13] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 3 context: , client address: IPv4 172.21.15.137:40764 2024-05-21T12:29:13.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:13] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:13.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:13] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:40764 2024-05-21T12:29:13.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:29:13.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:13.105831] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:29:13.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:13] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 , client address: IPv4 172.21.15.137:40764 2024-05-21T12:29:14.016 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:14.016 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:14.016 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage36 2024-05-21T12:29:14.016 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage36 --load-balancing-group 1 2024-05-21T12:29:14.093 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:14Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:14.815 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 16 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:15.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:14 smithi045 ceph-mon[30165]: pgmap v399: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 2.2 KiB/s wr, 77 op/s 2024-05-21T12:29:15.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:14] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 1 context: , client address: IPv4 172.21.15.137:58196 2024-05-21T12:29:15.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:14] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:15.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:14] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:58196 2024-05-21T12:29:15.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:29:15.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:14.800663] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:29:15.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:14] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 , client address: IPv4 172.21.15.137:58196 2024-05-21T12:29:15.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:14 smithi072 ceph-mon[33912]: pgmap v399: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 2.2 KiB/s wr, 77 op/s 2024-05-21T12:29:15.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:14] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:15.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:14] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:15.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:29:15.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:14.825770] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:29:15.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:14] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 2024-05-21T12:29:15.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:14 smithi092 ceph-mon[33665]: pgmap v399: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 2.2 KiB/s wr, 77 op/s 2024-05-21T12:29:15.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:14 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:14] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:15.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:14 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:14] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:15.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:14 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:29:15.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:14 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:14.826738] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:29:15.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:14 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:14] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 2024-05-21T12:29:15.733 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:16.058 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:16.058 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage37 2024-05-21T12:29:16.058 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage37 --load-balancing-group 2 2024-05-21T12:29:16.059 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:15Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:16.572 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 17 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:16 smithi045 ceph-mon[30165]: pgmap v400: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s rd, 1.6 KiB/s wr, 57 op/s 2024-05-21T12:29:16.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:16 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:16] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 2 context: , client address: IPv4 172.21.15.137:51558 2024-05-21T12:29:16.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:16 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:16] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:16.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:16 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:16] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:51558 2024-05-21T12:29:16.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:16 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:16] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:29:16.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:16 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:16.558301] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:29:16.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:16 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:16] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 , client address: IPv4 172.21.15.137:51558 2024-05-21T12:29:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:16 smithi072 ceph-mon[33912]: pgmap v400: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s rd, 1.6 KiB/s wr, 57 op/s 2024-05-21T12:29:16.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:16 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:16] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:16.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:16 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:16] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:16.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:16 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:16] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:29:16.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:16 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:16.581130] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:29:16.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:16 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:16] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 2024-05-21T12:29:17.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:16 smithi092 ceph-mon[33665]: pgmap v400: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s rd, 1.6 KiB/s wr, 57 op/s 2024-05-21T12:29:17.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:16 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:16] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:17.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:16 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:16] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:17.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:16 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:16] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:29:17.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:16 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:16.583980] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:29:17.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:16 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:16] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 2024-05-21T12:29:17.418 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:17.418 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:17.418 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage38 2024-05-21T12:29:17.418 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage38 --load-balancing-group 3 2024-05-21T12:29:17.519 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:17Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:18.271 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 18 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:18.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:18] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 3 context: , client address: IPv4 172.21.15.137:55628 2024-05-21T12:29:18.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:18] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:29:18.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:18] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 78,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:29:18.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:18] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:29:18.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:18] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:29:18.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:18] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:55628 2024-05-21T12:29:18.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:18] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:29:18.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:18.255298] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:29:18.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:18] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using UUID d826b447-494f-472c-bb8e-d88c0562fa80 , client address: IPv4 172.21.15.137:55628 2024-05-21T12:29:18.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:18 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:18] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:29:18.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:18 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:18] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:18.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:18 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:18] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:29:18.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:18 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:18.281288] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:29:18.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:18 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:18] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 2024-05-21T12:29:18.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:18 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:18] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:29:18.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:18 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:18] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:18.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:18 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:18] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:29:18.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:18 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:18.280179] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:29:18.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:18 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:18] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 2024-05-21T12:29:19.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-mon[30165]: pgmap v401: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 2.2 KiB/s wr, 67 op/s 2024-05-21T12:29:19.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:18 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2697230593' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:19.152 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:19.152 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:19.152 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage39 2024-05-21T12:29:19.152 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage39 --load-balancing-group 1 2024-05-21T12:29:19.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:18 smithi072 ceph-mon[33912]: pgmap v401: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 2.2 KiB/s wr, 67 op/s 2024-05-21T12:29:19.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:18 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2697230593' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:19.250 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:19Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:19.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:18 smithi092 ceph-mon[33665]: pgmap v401: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 2.2 KiB/s wr, 67 op/s 2024-05-21T12:29:19.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:18 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2697230593' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:19.975 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 19 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:20.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:19] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:20.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:19] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:20.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:19] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:29:20.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:19.989025] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:29:20.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:19 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:19] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 2024-05-21T12:29:20.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:19] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 1 context: , client address: IPv4 172.21.15.137:34424 2024-05-21T12:29:20.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:19] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:20.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:19] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:34424 2024-05-21T12:29:20.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:19] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:29:20.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:19.960180] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:29:20.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:19 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:19] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 , client address: IPv4 172.21.15.137:34424 2024-05-21T12:29:20.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:19] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:29:20.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:19] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:20.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:19] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:29:20.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:19.985200] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:29:20.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:19] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 2024-05-21T12:29:20.829 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:20.829 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:20.829 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage40 2024-05-21T12:29:20.829 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode2 --rbd-pool mypool --rbd-image myimage40 --load-balancing-group 2 2024-05-21T12:29:20.919 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:20Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:21.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:20 smithi045 ceph-mon[30165]: pgmap v402: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 1.4 KiB/s wr, 44 op/s 2024-05-21T12:29:21.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:20 smithi072 ceph-mon[33912]: pgmap v402: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 1.4 KiB/s wr, 44 op/s 2024-05-21T12:29:21.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:20 smithi092 ceph-mon[33665]: pgmap v402: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 1.4 KiB/s wr, 44 op/s 2024-05-21T12:29:21.652 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 20 to nqn.2016-06.io.spdk:cnode2: Successful 2024-05-21T12:29:21.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:21 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:21] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:21.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:21 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:21] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:21.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:21 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:21] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:29:21.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:21 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:21.665830] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:29:21.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:21 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:21] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c 2024-05-21T12:29:22.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:21 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:21] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:29:22.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:21 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:21] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:22.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:21 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:21] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:29:22.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:21 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:21.660492] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:29:22.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:21 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:21] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c 2024-05-21T12:29:22.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:21 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:21] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode2, ana group 2 context: , client address: IPv4 172.21.15.137:59116 2024-05-21T12:29:22.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:21 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:21] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:22.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:21 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:21] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:59116 2024-05-21T12:29:22.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:21 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:21] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:29:22.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:21 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:21.636994] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:29:22.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:21 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:21] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using UUID 4d993843-ec8c-481b-acef-898a1c83418c , client address: IPv4 172.21.15.137:59116 2024-05-21T12:29:22.513 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:22.513 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:29:22.513 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode3 2024-05-21T12:29:22.513 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 20 2024-05-21T12:29:22.514 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:22.514 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage41 2024-05-21T12:29:22.514 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage41 --load-balancing-group 3 2024-05-21T12:29:22.599 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:22Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:23.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:22 smithi045 ceph-mon[30165]: pgmap v403: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:23.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:22 smithi072 ceph-mon[33912]: pgmap v403: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:23.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:22 smithi092 ceph-mon[33665]: pgmap v403: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:23.328 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 1 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:23.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:23] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 3 context: , client address: IPv4 172.21.15.137:55712 2024-05-21T12:29:23.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:23] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:23.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:23] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:55712 2024-05-21T12:29:23.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:23] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:29:23.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:23.313512] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:29:23.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:23] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 , client address: IPv4 172.21.15.137:55712 2024-05-21T12:29:23.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:23 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:23] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:23.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:23 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:23] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:23.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:23 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:23] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:29:23.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:23 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:23.336669] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:29:23.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:23 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:23] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 2024-05-21T12:29:23.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:23 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:23] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:23.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:23 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:23] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:23.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:23 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:23] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:29:23.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:23 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:23.340824] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:29:23.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:23 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:23] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 2024-05-21T12:29:24.157 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:24.157 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:24.157 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage42 2024-05-21T12:29:24.157 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage42 --load-balancing-group 1 2024-05-21T12:29:24.237 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:24Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:24.961 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 2 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:25.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:24 smithi045 ceph-mon[30165]: pgmap v404: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:25.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:24] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 1 context: , client address: IPv4 172.21.15.137:37836 2024-05-21T12:29:25.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:24] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:25.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:24] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:37836 2024-05-21T12:29:25.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:24] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:29:25.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:24.946368] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:29:25.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:24 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:24] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 , client address: IPv4 172.21.15.137:37836 2024-05-21T12:29:25.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:24 smithi072 ceph-mon[33912]: pgmap v404: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:25.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:24] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:25.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:24] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:25.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:24] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:29:25.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:24.972291] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:29:25.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:24 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:24] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 2024-05-21T12:29:25.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:24 smithi092 ceph-mon[33665]: pgmap v404: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:25.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:24] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:25.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:24] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:25.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:24] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:29:25.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:24.973030] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:29:25.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:24] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 2024-05-21T12:29:25.832 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:25.832 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:25.832 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage43 2024-05-21T12:29:25.832 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage43 --load-balancing-group 2 2024-05-21T12:29:25.907 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:25Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:26.668 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 3 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:27.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:26 smithi092 ceph-mon[33665]: pgmap v405: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:27.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:26 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:26] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:27.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:26 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:26] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:27.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:26 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:26] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:29:27.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:26 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:26.677450] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:29:27.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:26 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:26] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 2024-05-21T12:29:27.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:26 smithi045 ceph-mon[30165]: pgmap v405: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:27.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:26] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 2 context: , client address: IPv4 172.21.15.137:35752 2024-05-21T12:29:27.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:26] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:27.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:26] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:35752 2024-05-21T12:29:27.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:26] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:29:27.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:26.654590] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:29:27.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:26 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:26] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 , client address: IPv4 172.21.15.137:35752 2024-05-21T12:29:27.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:26 smithi072 ceph-mon[33912]: pgmap v405: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:27.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:26 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:26] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:27.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:26 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:26] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:27.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:26 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:26] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:29:27.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:26 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:26.680555] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:29:27.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:26 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:26] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 2024-05-21T12:29:27.567 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:27.567 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:27.568 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage44 2024-05-21T12:29:27.568 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage44 --load-balancing-group 3 2024-05-21T12:29:27.644 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:27Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:28.345 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 4 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:28] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 3 context: , client address: IPv4 172.21.15.137:49430 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:28] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:28] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 88,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:28] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:28] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:28] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:49430 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:28] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:28.328552] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:29:28.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:28] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f , client address: IPv4 172.21.15.137:49430 2024-05-21T12:29:28.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:28 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:28] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:28.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:28 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:28] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:28.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:28 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:28] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:29:28.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:28 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:28.355938] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:29:28.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:28 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:28] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f 2024-05-21T12:29:28.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:28 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:28] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:28.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:28 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:28] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:28.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:28 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:28] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:29:28.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:28 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:28.358365] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:29:28.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:28 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:28] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f 2024-05-21T12:29:29.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-mon[30165]: pgmap v406: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 1.8 KiB/s wr, 60 op/s 2024-05-21T12:29:29.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:28 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1737140066' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:29.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:28 smithi072 ceph-mon[33912]: pgmap v406: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 1.8 KiB/s wr, 60 op/s 2024-05-21T12:29:29.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:28 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1737140066' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:29.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:28 smithi092 ceph-mon[33665]: pgmap v406: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 1.8 KiB/s wr, 60 op/s 2024-05-21T12:29:29.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:28 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1737140066' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:29.255 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:29.256 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:29.256 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage45 2024-05-21T12:29:29.256 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage45 --load-balancing-group 1 2024-05-21T12:29:29.328 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:29Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:29.983 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 5 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:30.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:29] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:30.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:29] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:30.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:29] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:29:30.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:29.996068] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:29:30.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:29 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:29] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 2024-05-21T12:29:30.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:29] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 1 context: , client address: IPv4 172.21.15.137:39124 2024-05-21T12:29:30.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:29] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:30.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:29] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:39124 2024-05-21T12:29:30.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:29] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:29:30.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:29.968693] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:29:30.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:29 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:29] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 , client address: IPv4 172.21.15.137:39124 2024-05-21T12:29:30.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:29] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:30.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:29] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:30.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:29] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:29:30.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:29.995551] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:29:30.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:29 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:29] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 2024-05-21T12:29:30.844 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:30.844 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:30.844 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage46 2024-05-21T12:29:30.844 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage46 --load-balancing-group 2 2024-05-21T12:29:30.929 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:30Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:31.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:30 smithi072 ceph-mon[33912]: pgmap v407: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s rd, 1.3 KiB/s wr, 55 op/s 2024-05-21T12:29:31.259 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:30 smithi092 ceph-mon[33665]: pgmap v407: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s rd, 1.3 KiB/s wr, 55 op/s 2024-05-21T12:29:31.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:30 smithi045 ceph-mon[30165]: pgmap v407: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s rd, 1.3 KiB/s wr, 55 op/s 2024-05-21T12:29:31.668 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 6 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:32.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:31 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:31] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:32.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:31 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:31] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:32.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:31 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:31] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:29:32.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:31 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:31.682094] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:29:32.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:31 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:31] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 2024-05-21T12:29:32.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:31 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:31] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 2 context: , client address: IPv4 172.21.15.137:36792 2024-05-21T12:29:32.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:31 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:31] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:32.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:31 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:31] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:36792 2024-05-21T12:29:32.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:31 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:31] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:29:32.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:31 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:31.652398] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:29:32.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:31 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:31] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 , client address: IPv4 172.21.15.137:36792 2024-05-21T12:29:32.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:31 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:31] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:32.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:31 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:31] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:32.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:31 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:31] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:29:32.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:31 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:31.680860] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:29:32.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:31 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:31] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 2024-05-21T12:29:32.546 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:32.546 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:32.546 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage47 2024-05-21T12:29:32.546 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage47 --load-balancing-group 3 2024-05-21T12:29:32.626 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:32Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:33.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:32 smithi072 ceph-mon[33912]: pgmap v408: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s rd, 1.5 KiB/s wr, 68 op/s 2024-05-21T12:29:33.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:32 smithi092 ceph-mon[33665]: pgmap v408: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s rd, 1.5 KiB/s wr, 68 op/s 2024-05-21T12:29:33.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:32 smithi045 ceph-mon[30165]: pgmap v408: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s rd, 1.5 KiB/s wr, 68 op/s 2024-05-21T12:29:33.391 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 7 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:33.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:33 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:33] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:33.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:33 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:33] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:33.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:33 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:33] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:29:33.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:33 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:33.400500] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:29:33.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:33 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:33] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 2024-05-21T12:29:33.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:33 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:33] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:33.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:33 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:33] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:33.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:33 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:33] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:29:33.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:33 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:33.402293] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:29:33.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:33 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:33] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 2024-05-21T12:29:33.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:33 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:33] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 3 context: , client address: IPv4 172.21.15.137:56702 2024-05-21T12:29:33.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:33 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:33] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:33.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:33 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:33] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:56702 2024-05-21T12:29:33.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:33 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:33] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:29:33.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:33 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:33.373823] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:29:33.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:33 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:33] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using UUID 0735e620-3968-43dc-a722-37465b1e9ce6 , client address: IPv4 172.21.15.137:56702 2024-05-21T12:29:34.257 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:34.258 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:34.258 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage48 2024-05-21T12:29:34.258 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage48 --load-balancing-group 1 2024-05-21T12:29:34.357 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:34Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:35.074 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 8 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:35.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:34 smithi072 ceph-mon[33912]: pgmap v409: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 2.1 KiB/s wr, 70 op/s 2024-05-21T12:29:35.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:35] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:35.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:35] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:35.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:35] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:29:35.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:35.084293] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:29:35.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:35] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 2024-05-21T12:29:35.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:34 smithi092 ceph-mon[33665]: pgmap v409: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 2.1 KiB/s wr, 70 op/s 2024-05-21T12:29:35.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:35] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:35.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:35] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:35.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:35] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:29:35.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:35.087680] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:29:35.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:35] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 2024-05-21T12:29:35.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:34 smithi045 ceph-mon[30165]: pgmap v409: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 2.1 KiB/s wr, 70 op/s 2024-05-21T12:29:35.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:35] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 1 context: , client address: IPv4 172.21.15.137:48148 2024-05-21T12:29:35.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:35] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:35.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:35] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:48148 2024-05-21T12:29:35.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:35] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:29:35.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:35.058555] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:29:35.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:35 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:35] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using UUID fbcff93c-1d7a-4216-9960-29e985da87e5 , client address: IPv4 172.21.15.137:48148 2024-05-21T12:29:35.948 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:35.948 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:35.948 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage49 2024-05-21T12:29:35.949 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage49 --load-balancing-group 2 2024-05-21T12:29:36.042 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:36Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:36.780 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 9 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:37.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:36 smithi045 ceph-mon[30165]: pgmap v410: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 1.4 KiB/s wr, 55 op/s 2024-05-21T12:29:37.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:36] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 2 context: , client address: IPv4 172.21.15.137:33788 2024-05-21T12:29:37.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:36] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:37.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:36] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:33788 2024-05-21T12:29:37.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:36] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:29:37.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:36.763479] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:29:37.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:36 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:36] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using UUID 9430b9c3-154d-4bdc-ac40-619f26451491 , client address: IPv4 172.21.15.137:33788 2024-05-21T12:29:37.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:36 smithi072 ceph-mon[33912]: pgmap v410: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 1.4 KiB/s wr, 55 op/s 2024-05-21T12:29:37.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:36] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:37.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:36] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:37.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:36] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:29:37.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:36.793405] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:29:37.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:36] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 2024-05-21T12:29:37.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:36 smithi092 ceph-mon[33665]: pgmap v410: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 1.4 KiB/s wr, 55 op/s 2024-05-21T12:29:37.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:36 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:36] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:37.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:36 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:36] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:37.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:36 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:36] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:29:37.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:36 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:36.791687] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:29:37.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:36 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:36] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 2024-05-21T12:29:37.667 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:37.667 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:37.668 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage50 2024-05-21T12:29:37.668 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage50 --load-balancing-group 3 2024-05-21T12:29:37.762 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:37Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:38.535 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 10 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:38.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:38] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 3 context: , client address: IPv4 172.21.15.137:53586 2024-05-21T12:29:38.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:38] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:29:38.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:38] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 98,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:29:38.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:38] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:29:38.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:38] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:29:38.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:38] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:53586 2024-05-21T12:29:38.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:38] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:29:38.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:38.520074] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:29:38.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:38] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da , client address: IPv4 172.21.15.137:53586 2024-05-21T12:29:38.857 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:38 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:38] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:38.857 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:38 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:38] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:38.857 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:38 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:38] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:29:38.857 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:38 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:38.546448] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:29:38.858 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:38 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:38] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da 2024-05-21T12:29:38.858 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:38] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:38.858 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:38] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:38.858 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:38] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:29:38.858 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:38.544796] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:29:38.858 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:38] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da 2024-05-21T12:29:39.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:38 smithi072 ceph-mon[33912]: pgmap v411: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 1.8 KiB/s wr, 57 op/s 2024-05-21T12:29:39.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:38 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/2309683200' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:39.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:38 smithi092 ceph-mon[33665]: pgmap v411: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 1.8 KiB/s wr, 57 op/s 2024-05-21T12:29:39.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:38 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/2309683200' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:39.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-mon[30165]: pgmap v411: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 1.8 KiB/s wr, 57 op/s 2024-05-21T12:29:39.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:38 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/2309683200' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:39.379 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:39.379 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:39.379 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage51 2024-05-21T12:29:39.379 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage51 --load-balancing-group 1 2024-05-21T12:29:39.482 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:39Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:40.212 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 11 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:40.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:40] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:40.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:40] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:40.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:29:40.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:40.220925] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:29:40.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:40] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd 2024-05-21T12:29:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:40] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 1 context: , client address: IPv4 172.21.15.137:54616 2024-05-21T12:29:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:40] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:40] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:54616 2024-05-21T12:29:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:29:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:40.197191] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:29:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:40] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd , client address: IPv4 172.21.15.137:54616 2024-05-21T12:29:40.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:40 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:40] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:40.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:40 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:40] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:40.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:40 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:29:40.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:40 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:40.223321] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:29:40.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:40 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:40] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd 2024-05-21T12:29:41.063 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:41.063 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:41.063 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage52 2024-05-21T12:29:41.063 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage52 --load-balancing-group 2 2024-05-21T12:29:41.149 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:41Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:41.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:40 smithi072 ceph-mon[33912]: pgmap v412: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:41.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:40 smithi092 ceph-mon[33665]: pgmap v412: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:41.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:40 smithi045 ceph-mon[30165]: pgmap v412: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s rd, 1.6 KiB/s wr, 60 op/s 2024-05-21T12:29:41.862 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 12 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:42.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:41] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:42.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:41] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:42.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:29:42.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:41.872054] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:29:42.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:41 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:41] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 2024-05-21T12:29:42.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:41] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:42.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:41] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:42.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:29:42.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:41.874245] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:29:42.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:41] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 2024-05-21T12:29:42.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:41] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 2 context: , client address: IPv4 172.21.15.137:49232 2024-05-21T12:29:42.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:41] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:42.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:41] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:49232 2024-05-21T12:29:42.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:29:42.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:41.847075] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:29:42.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:41] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 , client address: IPv4 172.21.15.137:49232 2024-05-21T12:29:42.716 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:42.717 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:42.717 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage53 2024-05-21T12:29:42.717 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage53 --load-balancing-group 3 2024-05-21T12:29:42.806 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:42Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:43.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:42 smithi072 ceph-mon[33912]: pgmap v413: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:29:43.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:42 smithi092 ceph-mon[33665]: pgmap v413: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:29:43.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:42 smithi045 ceph-mon[30165]: pgmap v413: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s rd, 1.7 KiB/s wr, 66 op/s 2024-05-21T12:29:43.561 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 13 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:43.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:43] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 3 context: , client address: IPv4 172.21.15.137:44488 2024-05-21T12:29:43.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:43] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:43.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:43] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:44488 2024-05-21T12:29:43.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:43] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:29:43.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:43.543952] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:29:43.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:43] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using UUID 34a70313-baa0-485f-bd37-d8df76585703 , client address: IPv4 172.21.15.137:44488 2024-05-21T12:29:43.877 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:43] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:43.878 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:43] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:43.878 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:43] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:29:43.878 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:43.570830] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:29:43.879 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:43] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 2024-05-21T12:29:43.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:43] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:43.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:43] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:43.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:43] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:29:43.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:43.568517] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:29:43.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:43] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 2024-05-21T12:29:44.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:43 smithi072 ceph-mon[33912]: pgmap v414: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:29:44.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:43 smithi092 ceph-mon[33665]: pgmap v414: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:29:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:43 smithi045 ceph-mon[30165]: pgmap v414: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 127 KiB/s rd, 2.1 KiB/s wr, 65 op/s 2024-05-21T12:29:44.443 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:44.443 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:44.443 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage54 2024-05-21T12:29:44.443 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage54 --load-balancing-group 1 2024-05-21T12:29:44.544 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:44Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:45.283 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 14 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:45.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:45] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 1 context: , client address: IPv4 172.21.15.137:57626 2024-05-21T12:29:45.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:45] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:45.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:45] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:57626 2024-05-21T12:29:45.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:45] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:29:45.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:45.268364] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:29:45.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:45] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 , client address: IPv4 172.21.15.137:57626 2024-05-21T12:29:45.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:45] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:45.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:45] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:45.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:45] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:29:45.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:45.292892] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:29:45.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:45] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 2024-05-21T12:29:45.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:45] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:45.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:45] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:45.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:45] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:29:45.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:45.294689] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:29:45.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:45] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 2024-05-21T12:29:46.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:45 smithi072 ceph-mon[33912]: pgmap v415: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s rd, 1.5 KiB/s wr, 58 op/s 2024-05-21T12:29:46.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:45 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:29:46.211 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:46.211 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:46.211 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage55 2024-05-21T12:29:46.211 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage55 --load-balancing-group 2 2024-05-21T12:29:46.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:45 smithi092 ceph-mon[33665]: pgmap v415: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s rd, 1.5 KiB/s wr, 58 op/s 2024-05-21T12:29:46.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:45 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:29:46.294 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:46Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:46.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:45 smithi045 ceph-mon[30165]: pgmap v415: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s rd, 1.5 KiB/s wr, 58 op/s 2024-05-21T12:29:46.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:45 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:29:47.019 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 15 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:47.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:46 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:29:47.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:46 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-05-21T12:29:47.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:46 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi045 to 6498M 2024-05-21T12:29:47.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:46 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:29:47.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:46 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:29:47.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:46 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:29:47.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:46 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:29:47.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:47] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:47.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:47] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:47.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:47] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:29:47.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:47.029326] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:29:47.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:47 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:47] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b 2024-05-21T12:29:47.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:46 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:29:47.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:46 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-05-21T12:29:47.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:46 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi045 to 6498M 2024-05-21T12:29:47.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:46 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:29:47.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:46 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:29:47.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:46 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:29:47.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:46 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:29:47.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:47] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:47.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:47] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:47.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:47] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:29:47.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:47.030799] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:29:47.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:47] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b 2024-05-21T12:29:47.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-05-21T12:29:47.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-05-21T12:29:47.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi045 to 6498M 2024-05-21T12:29:47.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:29:47.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:29:47.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:29:47.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:29:47.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:46] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 2 context: , client address: IPv4 172.21.15.137:45620 2024-05-21T12:29:47.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:46] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:47.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:46] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:45620 2024-05-21T12:29:47.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:46 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:46] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:29:47.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:47.001371] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:29:47.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:47 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:47] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using UUID 37f162b2-e7c8-44a7-b545-84d228432f2b , client address: IPv4 172.21.15.137:45620 2024-05-21T12:29:47.889 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:47.889 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:47.889 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage56 2024-05-21T12:29:47.889 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage56 --load-balancing-group 3 2024-05-21T12:29:47.978 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:47Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:48.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:47 smithi072 ceph-mon[33912]: pgmap v416: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 2.1 KiB/s wr, 68 op/s 2024-05-21T12:29:48.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:47 smithi092 ceph-mon[33665]: pgmap v416: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 2.1 KiB/s wr, 68 op/s 2024-05-21T12:29:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:47 smithi045 ceph-mon[30165]: pgmap v416: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 2.1 KiB/s wr, 68 op/s 2024-05-21T12:29:48.702 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 16 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:49.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:48 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1587224317' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:49.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:48] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:49.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:48] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:49.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:48] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:29:49.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:48.711609] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:29:49.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:48] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f 2024-05-21T12:29:49.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1587224317' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:49.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:48] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 3 context: , client address: IPv4 172.21.15.137:38318 2024-05-21T12:29:49.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:48] INFO cephutils.py:45: nvme-show string: {"prefix":"nvme-gw show", "pool":"mypool", "group":""} 2024-05-21T12:29:49.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:48] INFO cephutils.py:47: reply "(0, b'{\n "epoch": 108,\n "pool": "mypool",\n "group": "",\n "num gws": 3,\n "Anagrp list": "[ 1 2 3 ]"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.a",\n "anagrp-id": 1,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: ACTIVE , 2: STANDBY , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.b",\n "anagrp-id": 2,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: ACTIVE , 3: STANDBY ,"\n}\n{\n "gw-id": "client.nvmeof.nvmeof.c",\n "anagrp-id": 3,\n "performed-full-startup": 1,\n "Availability": "AVAILABLE",\n "ana states": " 1: STANDBY , 2: STANDBY , 3: ACTIVE ,"\n}\n', '')" 2024-05-21T12:29:49.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:48] INFO cephutils.py:55: new_str : 1 2 3 2024-05-21T12:29:49.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:48] INFO cephutils.py:58: [1, 2, 3] 2024-05-21T12:29:49.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:48] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:38318 2024-05-21T12:29:49.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:48] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:29:49.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:48.685912] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:29:49.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:48] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f , client address: IPv4 172.21.15.137:38318 2024-05-21T12:29:49.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:48 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1587224317' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"nvme-gw show", "pool":"mypool", "group":""} : dispatch 2024-05-21T12:29:49.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:48] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:49.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:48] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:49.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:48] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:29:49.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:48.714226] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:29:49.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:48 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:48] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f 2024-05-21T12:29:49.580 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:49.580 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:49.580 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage57 2024-05-21T12:29:49.580 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage57 --load-balancing-group 1 2024-05-21T12:29:49.660 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:49Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:50.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:49 smithi072 ceph-mon[33912]: pgmap v417: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:29:50.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:49 smithi092 ceph-mon[33665]: pgmap v417: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:29:50.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:49 smithi045 ceph-mon[30165]: pgmap v417: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:29:50.374 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 17 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:50.665 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:50] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:50.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:50] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:50.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:50] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:29:50.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:50.388234] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:29:50.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:50 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:50] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca 2024-05-21T12:29:50.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:50] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:50] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:50] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:29:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:50.385632] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:29:50.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:50] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca 2024-05-21T12:29:50.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:50 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:50] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 1 context: , client address: IPv4 172.21.15.137:50502 2024-05-21T12:29:50.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:50 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:50] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:50.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:50 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:50] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:50502 2024-05-21T12:29:50.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:50 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:50] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:29:50.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:50 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:50.357997] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:29:50.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:50 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:50] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using UUID e7985387-2968-452b-9709-e37a1a8255ca , client address: IPv4 172.21.15.137:50502 2024-05-21T12:29:51.267 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:51.267 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:51.267 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage58 2024-05-21T12:29:51.268 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage58 --load-balancing-group 2 2024-05-21T12:29:51.349 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:51Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:52.048 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 18 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:52.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:51 smithi072 ceph-mon[33912]: pgmap v418: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:29:52.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:52] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:52.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:52] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:52.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:29:52.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:52.056672] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:29:52.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:52 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:52] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b 2024-05-21T12:29:52.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:51 smithi092 ceph-mon[33665]: pgmap v418: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:29:52.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:52] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:29:52.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:52] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:52.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:29:52.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:52.058134] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:29:52.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:52] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b 2024-05-21T12:29:52.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:51 smithi045 ceph-mon[30165]: pgmap v418: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:29:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:52] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 2 context: , client address: IPv4 172.21.15.137:43416 2024-05-21T12:29:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:52] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:52] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:43416 2024-05-21T12:29:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:29:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:52.031463] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:29:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:52] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b , client address: IPv4 172.21.15.137:43416 2024-05-21T12:29:52.925 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:52.925 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:52.925 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage59 2024-05-21T12:29:52.925 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage59 --load-balancing-group 3 2024-05-21T12:29:53.026 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:53Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:53.761 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 19 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:54.066 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:53 smithi045 ceph-mon[30165]: pgmap v419: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 2.1 KiB/s wr, 64 op/s 2024-05-21T12:29:54.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:53] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 3 context: , client address: IPv4 172.21.15.137:55910 2024-05-21T12:29:54.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:53] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:54.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:53] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:55910 2024-05-21T12:29:54.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:29:54.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:53.744899] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:29:54.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:53] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using UUID d2218777-49ff-4c2d-84de-6d703a754573 , client address: IPv4 172.21.15.137:55910 2024-05-21T12:29:54.165 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:53 smithi072 ceph-mon[33912]: pgmap v419: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 2.1 KiB/s wr, 64 op/s 2024-05-21T12:29:54.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:53] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:54.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:53] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:54.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:29:54.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:53.772894] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:29:54.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:53] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:29:54.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:53 smithi092 ceph-mon[33665]: pgmap v419: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 2.1 KiB/s wr, 64 op/s 2024-05-21T12:29:54.252 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:29:54.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:54.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:29:54.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:53.769517] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:29:54.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:29:54.660 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:54.660 INFO:tasks.workunit.client.3.smithi137.stderr:+ for ns in $(seq 1 $NVMEOF_NAMESPACES_COUNT) 2024-05-21T12:29:54.660 INFO:tasks.workunit.client.3.smithi137.stderr:+ image=myimage60 2024-05-21T12:29:54.660 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 namespace add --subsystem nqn.2016-06.io.spdk:cnode3 --rbd-pool mypool --rbd-image myimage60 --load-balancing-group 1 2024-05-21T12:29:54.749 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:54Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:55.469 INFO:tasks.workunit.client.3.smithi137.stdout:Adding namespace 20 to nqn.2016-06.io.spdk:cnode3: Successful 2024-05-21T12:29:55.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:55.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:55.752 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:29:55.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:55.476877] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:29:55.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:29:55.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:55] INFO grpc.py:914: Received request to add a namespace to nqn.2016-06.io.spdk:cnode3, ana group 1 context: , client address: IPv4 172.21.15.137:39164 2024-05-21T12:29:55.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:55] INFO cephutils.py:38: Caching response of the monitor: [1, 2, 3] 2024-05-21T12:29:55.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:55] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist, client address: IPv4 172.21.15.137:39164 2024-05-21T12:29:55.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:55] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:29:55.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [2024-05-21 12:29:55.452779] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:29:55.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:55 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:55] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using UUID 77f81441-23a5-44ee-985c-f37050ecffc6 , client address: IPv4 172.21.15.137:39164 2024-05-21T12:29:55.915 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:55 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:55] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:29:55.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:55 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:55] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:29:55.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:55 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:55] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:29:55.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:55 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:29:55.481202] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:29:55.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:55 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:55] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:29:56.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:55 smithi092 ceph-mon[33665]: pgmap v420: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:56.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:55 smithi045 ceph-mon[30165]: pgmap v420: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:56.354 INFO:tasks.workunit.client.3.smithi137.stderr:+ (( image_index++ )) 2024-05-21T12:29:56.354 INFO:tasks.workunit.client.3.smithi137.stderr:+ list_subsystems 2024-05-21T12:29:56.355 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:29:56.355 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.45 2024-05-21T12:29:56.355 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 --format json subsystem list 2024-05-21T12:29:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:55 smithi072 ceph-mon[33912]: pgmap v420: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s rd, 1.5 KiB/s wr, 54 op/s 2024-05-21T12:29:56.446 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:56Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout:{ 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout: "error_message": "Success", 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout: "subsystems": [ 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode1", 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph40086972576734", 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:29:57.164 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 1, 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 2040, 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode2", 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph77937083268835", 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 1, 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 2040, 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:29:57.165 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode3", 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph15464245260738", 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 1, 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 2040, 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:29:57.166 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:29:57.167 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:29:57.167 INFO:tasks.workunit.client.3.smithi137.stdout: } 2024-05-21T12:29:57.167 INFO:tasks.workunit.client.3.smithi137.stdout: ], 2024-05-21T12:29:57.167 INFO:tasks.workunit.client.3.smithi137.stdout: "status": 0 2024-05-21T12:29:57.167 INFO:tasks.workunit.client.3.smithi137.stdout:} 2024-05-21T12:29:57.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:29:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:29:57] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:58730 2024-05-21T12:29:58.121 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:29:58.121 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.72 2024-05-21T12:29:58.121 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.72 --server-port 5500 --format json subsystem list 2024-05-21T12:29:58.197 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:58Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:29:58.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:29:57 smithi092 ceph-mon[33665]: pgmap v421: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 145 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:29:57 smithi045 ceph-mon[30165]: pgmap v421: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 145 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:29:57 smithi072 ceph-mon[33912]: pgmap v421: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 145 KiB/s rd, 2.2 KiB/s wr, 71 op/s 2024-05-21T12:29:58.907 INFO:tasks.workunit.client.3.smithi137.stdout:{ 2024-05-21T12:29:58.907 INFO:tasks.workunit.client.3.smithi137.stdout: "error_message": "Success", 2024-05-21T12:29:58.907 INFO:tasks.workunit.client.3.smithi137.stdout: "subsystems": [ 2024-05-21T12:29:58.907 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode1", 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph40086972576734", 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 2041, 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 4080, 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode2", 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:29:58.908 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph77937083268835", 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 2041, 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 4080, 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode3", 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph15464245260738", 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:29:58.909 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 2041, 2024-05-21T12:29:58.910 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 4080, 2024-05-21T12:29:58.910 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:29:58.910 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:29:58.910 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:29:58.910 INFO:tasks.workunit.client.3.smithi137.stdout: } 2024-05-21T12:29:58.910 INFO:tasks.workunit.client.3.smithi137.stdout: ], 2024-05-21T12:29:58.910 INFO:tasks.workunit.client.3.smithi137.stdout: "status": 0 2024-05-21T12:29:58.910 INFO:tasks.workunit.client.3.smithi137.stdout:} 2024-05-21T12:29:59.165 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:29:58 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:29:58] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:42876 2024-05-21T12:29:59.747 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in "${!gateway_ips[@]}" 2024-05-21T12:29:59.747 INFO:tasks.workunit.client.3.smithi137.stderr:+ ip=172.21.15.92 2024-05-21T12:29:59.747 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.92 --server-port 5500 --format json subsystem list 2024-05-21T12:29:59.851 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:29:59Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:30:00.519 INFO:tasks.workunit.client.3.smithi137.stdout:{ 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "error_message": "Success", 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "subsystems": [ 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode1", 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph40086972576734", 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 4081, 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 6120, 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:30:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode2", 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph77937083268835", 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 4081, 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 6120, 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: }, 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: { 2024-05-21T12:30:00.521 INFO:tasks.workunit.client.3.smithi137.stdout: "nqn": "nqn.2016-06.io.spdk:cnode3", 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "enable_ha": true, 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "serial_number": "Ceph15464245260738", 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "model_number": "Ceph bdev Controller", 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "min_cntlid": 4081, 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "max_cntlid": 6120, 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "namespace_count": 20, 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "subtype": "NVMe", 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "max_namespaces": 256 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: } 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: ], 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout: "status": 0 2024-05-21T12:30:00.522 INFO:tasks.workunit.client.3.smithi137.stdout:} 2024-05-21T12:30:00.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:00 smithi072 ceph-mon[33912]: pgmap v422: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:30:00.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:00 smithi072 ceph-mon[33912]: overall HEALTH_OK 2024-05-21T12:30:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:00 smithi092 ceph-mon[33665]: pgmap v422: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:30:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:00 smithi092 ceph-mon[33665]: overall HEALTH_OK 2024-05-21T12:30:00.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:30:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:30:00] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:52412 2024-05-21T12:30:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:00 smithi045 ceph-mon[30165]: pgmap v422: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.6 KiB/s wr, 59 op/s 2024-05-21T12:30:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:00 smithi045 ceph-mon[30165]: overall HEALTH_OK 2024-05-21T12:30:01.433 INFO:tasks.workunit.client.3.smithi137.stderr:++ seq 1 3 2024-05-21T12:30:01.434 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:30:01.434 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:01.434 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 --format plain namespace list --subsystem nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:01.515 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:30:01Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:30:02.381 INFO:tasks.workunit.client.3.smithi137.stdout:Namespaces in subsystem nqn.2016-06.io.spdk:cnode1: 2024-05-21T12:30:02.381 INFO:tasks.workunit.client.3.smithi137.stdout: NSID Bdev RBD RBD Image Block UUID Load R/W IOs R/W MBs Read MBs Write MBs 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: Name Pool Image Size Size Balancing per per per per 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: Group second second second second 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: 1 bdev_1ac8703e-ed04- mypool myimage1 8 GiB 512 Bytes 1ac8703e-ed04-493b- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: 493b-ba52-c73153370d7b ba52-c73153370d7b 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: 2 bdev_fd2b5ba2-2f5c- mypool myimage2 8 GiB 512 Bytes fd2b5ba2-2f5c-46fb- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: 46fb-8916-674d2cbb477b 8916-674d2cbb477b 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: 3 bdev_3185e485-c8a5- mypool myimage3 8 GiB 512 Bytes 3185e485-c8a5-4af2- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: 4af2-ae8e-15d7100dcdf0 ae8e-15d7100dcdf0 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: 4 bdev_74c74c54-3d4e- mypool myimage4 8 GiB 512 Bytes 74c74c54-3d4e-4485- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.382 INFO:tasks.workunit.client.3.smithi137.stdout: 4485-ba42-2b584b6fa81e ba42-2b584b6fa81e 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 5 bdev_2363a1af-0d49- mypool myimage5 8 GiB 512 Bytes 2363a1af-0d49-4f75- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 4f75-81f0-8b985c6c90d2 81f0-8b985c6c90d2 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 6 bdev_e349eac9-5fb1- mypool myimage6 8 GiB 512 Bytes e349eac9-5fb1-4829- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 4829-8523-49a423678213 8523-49a423678213 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 7 bdev_95c9bca2-e876- mypool myimage7 8 GiB 512 Bytes 95c9bca2-e876-4ec5- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 4ec5-8958-2e2ca7e60371 8958-2e2ca7e60371 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 8 bdev_b87a5ac5-b293- mypool myimage8 8 GiB 512 Bytes b87a5ac5-b293-4485- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 4485-b785-2a12b9a685af b785-2a12b9a685af 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 9 bdev_867f5e00-8d71- mypool myimage9 8 GiB 512 Bytes 867f5e00-8d71-4aae- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 4aae-b31a-dc9fb010b3b6 b31a-dc9fb010b3b6 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 10 bdev_cba00c90-3501- mypool myimage10 8 GiB 512 Bytes cba00c90-3501-4d43- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.383 INFO:tasks.workunit.client.3.smithi137.stdout: 4d43-b5c2-017aac10a02d b5c2-017aac10a02d 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 11 bdev_f4c0a98c-8cd4- mypool myimage11 8 GiB 512 Bytes f4c0a98c-8cd4-427d- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 427d-b870-36c81cc89c85 b870-36c81cc89c85 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 12 bdev_ee1e99bf-15bd- mypool myimage12 8 GiB 512 Bytes ee1e99bf-15bd-4039- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 4039-abfd-8e40153fe06e abfd-8e40153fe06e 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 13 bdev_513f8027-44a0- mypool myimage13 8 GiB 512 Bytes 513f8027-44a0-43dc- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 43dc-82c0-6df74017b797 82c0-6df74017b797 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 14 bdev_71b9a6c7-44e0- mypool myimage14 8 GiB 512 Bytes 71b9a6c7-44e0-4e40- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 4e40-95a2-a76947588015 95a2-a76947588015 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 15 bdev_346f90e2-ed39- mypool myimage15 8 GiB 512 Bytes 346f90e2-ed39-4090- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 4090-8599-e1283c82d3ad 8599-e1283c82d3ad 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 16 bdev_d0f0d22d-353b- mypool myimage16 8 GiB 512 Bytes d0f0d22d-353b-42cf- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 42cf-a21e-ff56cdf02783 a21e-ff56cdf02783 2024-05-21T12:30:02.384 INFO:tasks.workunit.client.3.smithi137.stdout: 17 bdev_6f2d5759-26b1- mypool myimage17 8 GiB 512 Bytes 6f2d5759-26b1-4a43- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.385 INFO:tasks.workunit.client.3.smithi137.stdout: 4a43-a40f-9a4ef53b5a93 a40f-9a4ef53b5a93 2024-05-21T12:30:02.385 INFO:tasks.workunit.client.3.smithi137.stdout: 18 bdev_1af254a8-e8d6- mypool myimage18 8 GiB 512 Bytes 1af254a8-e8d6-4e20- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.385 INFO:tasks.workunit.client.3.smithi137.stdout: 4e20-9188-384ceaa4505a 9188-384ceaa4505a 2024-05-21T12:30:02.385 INFO:tasks.workunit.client.3.smithi137.stdout: 19 bdev_df7d58b1-d12c- mypool myimage19 8 GiB 512 Bytes df7d58b1-d12c-4a99- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.385 INFO:tasks.workunit.client.3.smithi137.stdout: 4a99-8a5c-768adcfd252b 8a5c-768adcfd252b 2024-05-21T12:30:02.385 INFO:tasks.workunit.client.3.smithi137.stdout: 20 bdev_a6235233-7e8c- mypool myimage20 8 GiB 512 Bytes a6235233-7e8c-4b35- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:02.385 INFO:tasks.workunit.client.3.smithi137.stdout: 4b35-8493-fd5bfbe280cc 8493-fd5bfbe280cc 2024-05-21T12:30:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:02 smithi045 ceph-mon[30165]: pgmap v423: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:30:02.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:02 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:30:02] INFO grpc.py:1232: Received request to list all namespaces for nqn.2016-06.io.spdk:cnode1, context: , client address: IPv4 172.21.15.137:51310 2024-05-21T12:30:02.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:02 smithi072 ceph-mon[33912]: pgmap v423: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:30:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:02 smithi092 ceph-mon[33665]: pgmap v423: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s rd, 1.5 KiB/s wr, 53 op/s 2024-05-21T12:30:03.242 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:30:03.243 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode2 2024-05-21T12:30:03.243 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 --format plain namespace list --subsystem nqn.2016-06.io.spdk:cnode2 2024-05-21T12:30:03.330 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:30:03Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:30:04.189 INFO:tasks.workunit.client.3.smithi137.stdout:Namespaces in subsystem nqn.2016-06.io.spdk:cnode2: 2024-05-21T12:30:04.189 INFO:tasks.workunit.client.3.smithi137.stdout: NSID Bdev RBD RBD Image Block UUID Load R/W IOs R/W MBs Read MBs Write MBs 2024-05-21T12:30:04.189 INFO:tasks.workunit.client.3.smithi137.stdout: Name Pool Image Size Size Balancing per per per per 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: Group second second second second 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 1 bdev_88372a05-54d9- mypool myimage21 8 GiB 512 Bytes 88372a05-54d9-4c17- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 4c17-af42-2b9c02991736 af42-2b9c02991736 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 2 bdev_b5a82937-0d41- mypool myimage22 8 GiB 512 Bytes b5a82937-0d41-4fa2- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 4fa2-8d4d-9f933c80f1ed 8d4d-9f933c80f1ed 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 3 bdev_4dbc9214-2d75- mypool myimage23 8 GiB 512 Bytes 4dbc9214-2d75-4dd3- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 4dd3-850f-ae87ff8b7ddc 850f-ae87ff8b7ddc 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 4 bdev_d9bfd1c8-81ea- mypool myimage24 8 GiB 512 Bytes d9bfd1c8-81ea-4eb9- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 4eb9-8603-5618e621e98b 8603-5618e621e98b 2024-05-21T12:30:04.190 INFO:tasks.workunit.client.3.smithi137.stdout: 5 bdev_5aaed34a-1696- mypool myimage25 8 GiB 512 Bytes 5aaed34a-1696-499e- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 499e-b0b0-fad88d740507 b0b0-fad88d740507 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 6 bdev_c21863b3-ea38- mypool myimage26 8 GiB 512 Bytes c21863b3-ea38-4777- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 4777-afc0-6a7011958143 afc0-6a7011958143 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 7 bdev_ea2d877e-1a43- mypool myimage27 8 GiB 512 Bytes ea2d877e-1a43-4674- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 4674-a681-a0d8bf464c34 a681-a0d8bf464c34 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 8 bdev_6f017e0c-1259- mypool myimage28 8 GiB 512 Bytes 6f017e0c-1259-41c2- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 41c2-aa4e-63e3a6d75015 aa4e-63e3a6d75015 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 9 bdev_deecdc0a-c85f- mypool myimage29 8 GiB 512 Bytes deecdc0a-c85f-47e5- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 47e5-a88d-af0d5182e6ea a88d-af0d5182e6ea 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 10 bdev_42686ed2-6784- mypool myimage30 8 GiB 512 Bytes 42686ed2-6784-4220- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.191 INFO:tasks.workunit.client.3.smithi137.stdout: 4220-a2d9-6fd164e3c2f3 a2d9-6fd164e3c2f3 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 11 bdev_6fc6c70a-1f65- mypool myimage31 8 GiB 512 Bytes 6fc6c70a-1f65-486d- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 486d-bdd6-09bf4df6f9a3 bdd6-09bf4df6f9a3 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 12 bdev_38942ad5-39c0- mypool myimage32 8 GiB 512 Bytes 38942ad5-39c0-4396- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 4396-85fa-87bbd7a3d770 85fa-87bbd7a3d770 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 13 bdev_97ec7eca-d2dc- mypool myimage33 8 GiB 512 Bytes 97ec7eca-d2dc-4254- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 4254-a4ca-12d17cc765b7 a4ca-12d17cc765b7 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 14 bdev_bd92ec20-9f67- mypool myimage34 8 GiB 512 Bytes bd92ec20-9f67-4d5e- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 4d5e-aa72-bc5eddab838a aa72-bc5eddab838a 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 15 bdev_f8bee28e-a7ba- mypool myimage35 8 GiB 512 Bytes f8bee28e-a7ba-43f6- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 43f6-9e41-90f574236e57 9e41-90f574236e57 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 16 bdev_b9f77c65-3e74- mypool myimage36 8 GiB 512 Bytes b9f77c65-3e74-4b5e- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 4b5e-9e44-0bbc82231e40 9e44-0bbc82231e40 2024-05-21T12:30:04.192 INFO:tasks.workunit.client.3.smithi137.stdout: 17 bdev_899d8ccd-f410- mypool myimage37 8 GiB 512 Bytes 899d8ccd-f410-47e8- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.193 INFO:tasks.workunit.client.3.smithi137.stdout: 47e8-9ef3-478a9d409408 9ef3-478a9d409408 2024-05-21T12:30:04.193 INFO:tasks.workunit.client.3.smithi137.stdout: 18 bdev_d826b447-494f- mypool myimage38 8 GiB 512 Bytes d826b447-494f-472c- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.193 INFO:tasks.workunit.client.3.smithi137.stdout: 472c-bb8e-d88c0562fa80 bb8e-d88c0562fa80 2024-05-21T12:30:04.193 INFO:tasks.workunit.client.3.smithi137.stdout: 19 bdev_d3d6400e-10dc- mypool myimage39 8 GiB 512 Bytes d3d6400e-10dc-4ec2- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.193 INFO:tasks.workunit.client.3.smithi137.stdout: 4ec2-96ec-18e93314b495 96ec-18e93314b495 2024-05-21T12:30:04.193 INFO:tasks.workunit.client.3.smithi137.stdout: 20 bdev_4d993843-ec8c- mypool myimage40 8 GiB 512 Bytes 4d993843-ec8c-481b- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:04.193 INFO:tasks.workunit.client.3.smithi137.stdout: 481b-acef-898a1c83418c acef-898a1c83418c 2024-05-21T12:30:04.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:30:04] INFO grpc.py:1232: Received request to list all namespaces for nqn.2016-06.io.spdk:cnode2, context: , client address: IPv4 172.21.15.137:47990 2024-05-21T12:30:04.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:04 smithi072 ceph-mon[33912]: pgmap v424: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-05-21T12:30:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:04 smithi092 ceph-mon[33665]: pgmap v424: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-05-21T12:30:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:04 smithi045 ceph-mon[30165]: pgmap v424: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-05-21T12:30:05.084 INFO:tasks.workunit.client.3.smithi137.stderr:+ for i in $(seq 1 $NVMEOF_SUBSYSTEMS_COUNT) 2024-05-21T12:30:05.085 INFO:tasks.workunit.client.3.smithi137.stderr:+ subsystem_nqn=nqn.2016-06.io.spdk:cnode3 2024-05-21T12:30:05.085 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo podman run -it quay.io/ceph/nvmeof-cli:1.2 --server-address 172.21.15.45 --server-port 5500 --format plain namespace list --subsystem nqn.2016-06.io.spdk:cnode3 2024-05-21T12:30:05.176 INFO:tasks.workunit.client.3.smithi137.stderr:time="2024-05-21T12:30:05Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2024-05-21T12:30:06.049 INFO:tasks.workunit.client.3.smithi137.stdout:Namespaces in subsystem nqn.2016-06.io.spdk:cnode3: 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: NSID Bdev RBD RBD Image Block UUID Load R/W IOs R/W MBs Read MBs Write MBs 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: Name Pool Image Size Size Balancing per per per per 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: Group second second second second 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: 1 bdev_514d58ae-41ac- mypool myimage41 8 GiB 512 Bytes 514d58ae-41ac-4a18- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: 4a18-bad6-06a6168f1234 bad6-06a6168f1234 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: 2 bdev_c2aaf5f6-e955- mypool myimage42 8 GiB 512 Bytes c2aaf5f6-e955-4853- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: 4853-b5d2-d4d9e8d86558 b5d2-d4d9e8d86558 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: 3 bdev_894b8b8c-3043- mypool myimage43 8 GiB 512 Bytes 894b8b8c-3043-469c- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: 469c-bfcf-b11efc460de6 bfcf-b11efc460de6 2024-05-21T12:30:06.050 INFO:tasks.workunit.client.3.smithi137.stdout: 4 bdev_7a09ebd3-49a1- mypool myimage44 8 GiB 512 Bytes 7a09ebd3-49a1-4ee5- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 4ee5-8e71-41364492b16f 8e71-41364492b16f 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 5 bdev_2b67ac15-cf2c- mypool myimage45 8 GiB 512 Bytes 2b67ac15-cf2c-4cc6- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 4cc6-a1fc-8ee82ca7f327 a1fc-8ee82ca7f327 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 6 bdev_ae876ba4-a37b- mypool myimage46 8 GiB 512 Bytes ae876ba4-a37b-4c47- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 4c47-90ea-a61c0ae1dc39 90ea-a61c0ae1dc39 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 7 bdev_0735e620-3968- mypool myimage47 8 GiB 512 Bytes 0735e620-3968-43dc- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 43dc-a722-37465b1e9ce6 a722-37465b1e9ce6 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 8 bdev_fbcff93c-1d7a- mypool myimage48 8 GiB 512 Bytes fbcff93c-1d7a-4216- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 4216-9960-29e985da87e5 9960-29e985da87e5 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 9 bdev_9430b9c3-154d- mypool myimage49 8 GiB 512 Bytes 9430b9c3-154d-4bdc- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 4bdc-ac40-619f26451491 ac40-619f26451491 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 10 bdev_1e1ea26a-6294- mypool myimage50 8 GiB 512 Bytes 1e1ea26a-6294-47e4- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.051 INFO:tasks.workunit.client.3.smithi137.stdout: 47e4-b3d6-3d11171843da b3d6-3d11171843da 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 11 bdev_ffd15d0b-cb83- mypool myimage51 8 GiB 512 Bytes ffd15d0b-cb83-4048- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 4048-beb1-b65319b34ddd beb1-b65319b34ddd 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 12 bdev_8bad29f1-fb93- mypool myimage52 8 GiB 512 Bytes 8bad29f1-fb93-4936- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 4936-a7b2-067cb9715706 a7b2-067cb9715706 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 13 bdev_34a70313-baa0- mypool myimage53 8 GiB 512 Bytes 34a70313-baa0-485f- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 485f-bd37-d8df76585703 bd37-d8df76585703 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 14 bdev_7b5d8f0c-b651- mypool myimage54 8 GiB 512 Bytes 7b5d8f0c-b651-4f87- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 4f87-a24a-8700ba6f4ee8 a24a-8700ba6f4ee8 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 15 bdev_37f162b2-e7c8- mypool myimage55 8 GiB 512 Bytes 37f162b2-e7c8-44a7- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 44a7-b545-84d228432f2b b545-84d228432f2b 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 16 bdev_4a4c30b3-7624- mypool myimage56 8 GiB 512 Bytes 4a4c30b3-7624-4e49- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.052 INFO:tasks.workunit.client.3.smithi137.stdout: 4e49-b965-914eca74bb8f b965-914eca74bb8f 2024-05-21T12:30:06.053 INFO:tasks.workunit.client.3.smithi137.stdout: 17 bdev_e7985387-2968- mypool myimage57 8 GiB 512 Bytes e7985387-2968-452b- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.053 INFO:tasks.workunit.client.3.smithi137.stdout: 452b-9709-e37a1a8255ca 9709-e37a1a8255ca 2024-05-21T12:30:06.053 INFO:tasks.workunit.client.3.smithi137.stdout: 18 bdev_add34d0b-baf5- mypool myimage58 8 GiB 512 Bytes add34d0b-baf5-4a4e- 2 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.053 INFO:tasks.workunit.client.3.smithi137.stdout: 4a4e-bfa3-90b32af15b9b bfa3-90b32af15b9b 2024-05-21T12:30:06.053 INFO:tasks.workunit.client.3.smithi137.stdout: 19 bdev_d2218777-49ff- mypool myimage59 8 GiB 512 Bytes d2218777-49ff-4c2d- 3 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.053 INFO:tasks.workunit.client.3.smithi137.stdout: 4c2d-84de-6d703a754573 84de-6d703a754573 2024-05-21T12:30:06.053 INFO:tasks.workunit.client.3.smithi137.stdout: 20 bdev_77f81441-23a5- mypool myimage60 8 GiB 512 Bytes 77f81441-23a5-44ee- 1 unlimited unlimited unlimited unlimited 2024-05-21T12:30:06.053 INFO:tasks.workunit.client.3.smithi137.stdout: 44ee-985c-f37050ecffc6 985c-f37050ecffc6 2024-05-21T12:30:06.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:05 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:30:05] INFO grpc.py:1232: Received request to list all namespaces for nqn.2016-06.io.spdk:cnode3, context: , client address: IPv4 172.21.15.137:58704 2024-05-21T12:30:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:06 smithi072 ceph-mon[33912]: pgmap v425: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 767 B/s wr, 28 op/s 2024-05-21T12:30:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:06 smithi092 ceph-mon[33665]: pgmap v425: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 767 B/s wr, 28 op/s 2024-05-21T12:30:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:06 smithi045 ceph-mon[30165]: pgmap v425: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 767 B/s wr, 28 op/s 2024-05-21T12:30:06.961 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Subsystem setup done 2024-05-21T12:30:06.962 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Subsystem setup done' 2024-05-21T12:30:06.963 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-21T12:30:06.963 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.3/client.3/tmp 2024-05-21T12:30:07.031 INFO:tasks.workunit:Running workunits matching rbd/nvmeof_basic_tests.sh on client.3... 2024-05-21T12:30:07.032 INFO:tasks.workunit:Running workunit rbd/nvmeof_basic_tests.sh... 2024-05-21T12:30:07.032 DEBUG:teuthology.orchestra.run.smithi137:workunit test rbd/nvmeof_basic_tests.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && cd -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68fa3266016634745d991c9967c479c757966afe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="3" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.3 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.3 CEPH_MNT=/home/ubuntu/cephtest/mnt.3 RBD_IMAGE_PREFIX=myimage RBD_POOL=mypool timeout 3h /home/ubuntu/cephtest/clone.client.3/qa/workunits/rbd/nvmeof_basic_tests.sh 2024-05-21T12:30:07.087 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo modprobe nvme-fabrics 2024-05-21T12:30:07.112 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo modprobe nvme-tcp 2024-05-21T12:30:07.128 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo dnf reinstall nvme-cli -y 2024-05-21T12:30:07.646 INFO:tasks.workunit.client.3.smithi137.stdout:Last metadata expiration check: 0:17:32 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:30:07.768 INFO:tasks.workunit.client.3.smithi137.stdout:Dependencies resolved. 2024-05-21T12:30:07.769 INFO:tasks.workunit.client.3.smithi137.stdout:================================================================================ 2024-05-21T12:30:07.769 INFO:tasks.workunit.client.3.smithi137.stdout: Package Architecture Version Repository Size 2024-05-21T12:30:07.769 INFO:tasks.workunit.client.3.smithi137.stdout:================================================================================ 2024-05-21T12:30:07.769 INFO:tasks.workunit.client.3.smithi137.stdout:Reinstalling: 2024-05-21T12:30:07.769 INFO:tasks.workunit.client.3.smithi137.stdout: nvme-cli x86_64 2.9.1-2.el9 baseos 906 k 2024-05-21T12:30:07.769 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:30:07.769 INFO:tasks.workunit.client.3.smithi137.stdout:Transaction Summary 2024-05-21T12:30:07.769 INFO:tasks.workunit.client.3.smithi137.stdout:================================================================================ 2024-05-21T12:30:07.770 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:30:07.770 INFO:tasks.workunit.client.3.smithi137.stdout:Total download size: 906 k 2024-05-21T12:30:07.770 INFO:tasks.workunit.client.3.smithi137.stdout:Installed size: 5.2 M 2024-05-21T12:30:07.770 INFO:tasks.workunit.client.3.smithi137.stdout:Downloading Packages: 2024-05-21T12:30:08.273 INFO:tasks.workunit.client.3.smithi137.stdout:nvme-cli-2.9.1-2.el9.x86_64.rpm 2.0 MB/s | 906 kB 00:00 2024-05-21T12:30:08.273 INFO:tasks.workunit.client.3.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:30:08.273 INFO:tasks.workunit.client.3.smithi137.stdout:Total 1.8 MB/s | 906 kB 00:00 2024-05-21T12:30:08.291 INFO:tasks.workunit.client.3.smithi137.stdout:Running transaction check 2024-05-21T12:30:08.301 INFO:tasks.workunit.client.3.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:30:08.302 INFO:tasks.workunit.client.3.smithi137.stdout:Running transaction test 2024-05-21T12:30:08.389 INFO:tasks.workunit.client.3.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:30:08.389 INFO:tasks.workunit.client.3.smithi137.stdout:Running transaction 2024-05-21T12:30:08.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:08 smithi072 ceph-mon[33912]: pgmap v426: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 767 B/s wr, 27 op/s 2024-05-21T12:30:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:08 smithi092 ceph-mon[33665]: pgmap v426: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 767 B/s wr, 27 op/s 2024-05-21T12:30:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:08 smithi045 ceph-mon[30165]: pgmap v426: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 767 B/s wr, 27 op/s 2024-05-21T12:30:08.832 INFO:tasks.workunit.client.3.smithi137.stdout: Preparing : 1/1 2024-05-21T12:30:08.943 INFO:tasks.workunit.client.3.smithi137.stdout: Reinstalling : nvme-cli-2.9.1-2.el9.x86_64 1/2 2024-05-21T12:30:09.243 INFO:tasks.workunit.client.3.smithi137.stdout: Running scriptlet: nvme-cli-2.9.1-2.el9.x86_64 1/2 2024-05-21T12:30:09.379 INFO:tasks.workunit.client.3.smithi137.stdout: Cleanup : nvme-cli-2.9.1-2.el9.x86_64 2/2 2024-05-21T12:30:10.145 INFO:tasks.workunit.client.3.smithi137.stdout: Running scriptlet: nvme-cli-2.9.1-2.el9.x86_64 2/2 2024-05-21T12:30:10.146 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : nvme-cli-2.9.1-2.el9.x86_64 1/2 2024-05-21T12:30:10.490 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : nvme-cli-2.9.1-2.el9.x86_64 2/2 2024-05-21T12:30:10.490 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:30:10.490 INFO:tasks.workunit.client.3.smithi137.stdout:Reinstalled: 2024-05-21T12:30:10.490 INFO:tasks.workunit.client.3.smithi137.stdout: nvme-cli-2.9.1-2.el9.x86_64 2024-05-21T12:30:10.490 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:30:10.490 INFO:tasks.workunit.client.3.smithi137.stdout:Complete! 2024-05-21T12:30:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:10 smithi072 ceph-mon[33912]: pgmap v427: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2 op/s 2024-05-21T12:30:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:10 smithi092 ceph-mon[33665]: pgmap v427: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2 op/s 2024-05-21T12:30:10.779 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo lsmod 2024-05-21T12:30:10.779 INFO:tasks.workunit.client.3.smithi137.stderr:+ grep nvme 2024-05-21T12:30:10.794 INFO:tasks.workunit.client.3.smithi137.stdout:nvme_tcp 49152 0 2024-05-21T12:30:10.794 INFO:tasks.workunit.client.3.smithi137.stdout:nvme_fabrics 40960 1 nvme_tcp 2024-05-21T12:30:10.794 INFO:tasks.workunit.client.3.smithi137.stdout:nvme 65536 5 2024-05-21T12:30:10.794 INFO:tasks.workunit.client.3.smithi137.stdout:nvme_core 221184 8 nvme_tcp,nvme,nvme_fabrics 2024-05-21T12:30:10.794 INFO:tasks.workunit.client.3.smithi137.stdout:nvme_common 24576 1 nvme_core 2024-05-21T12:30:10.794 INFO:tasks.workunit.client.3.smithi137.stdout:t10_pi 24576 2 sd_mod,nvme_core 2024-05-21T12:30:10.794 INFO:tasks.workunit.client.3.smithi137.stderr:+ nvme version 2024-05-21T12:30:10.795 INFO:tasks.workunit.client.3.smithi137.stdout:nvme version 2.9.1 (git 2.9.1) 2024-05-21T12:30:10.795 INFO:tasks.workunit.client.3.smithi137.stdout:libnvme version 1.9 (git 1.9) 2024-05-21T12:30:10.796 INFO:tasks.workunit.client.3.smithi137.stderr:+ source /etc/ceph/nvmeof.env 2024-05-21T12:30:10.796 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_GATEWAY_IP_ADDRESSES=172.21.15.45,172.21.15.72,172.21.15.92 2024-05-21T12:30:10.796 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_GATEWAY_NAMES=smithi045,smithi072,smithi092 2024-05-21T12:30:10.796 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_DEFAULT_GATEWAY_IP_ADDRESS=172.21.15.45 2024-05-21T12:30:10.796 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_CLI_IMAGE=quay.io/ceph/nvmeof-cli:1.2 2024-05-21T12:30:10.796 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_SUBSYSTEMS_PREFIX=nqn.2016-06.io.spdk:cnode 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_SUBSYSTEMS_COUNT=3 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_NAMESPACES_COUNT=20 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_PORT=4420 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stderr:++ NVMEOF_SRPORT=5500 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stderr:+ SPDK_CONTROLLER='Ceph bdev Controller' 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stderr:+ DISCOVERY_PORT=8009 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stderr:+ test_run disconnect_all 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Running test: disconnect_all' 2024-05-21T12:30:10.797 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Running test: disconnect_all 2024-05-21T12:30:10.798 INFO:tasks.workunit.client.3.smithi137.stderr:+ disconnect_all 2024-05-21T12:30:10.798 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo nvme disconnect-all 2024-05-21T12:30:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:10 smithi045 ceph-mon[30165]: pgmap v427: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2 op/s 2024-05-21T12:30:10.817 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme list 2024-05-21T12:30:10.829 INFO:tasks.workunit.client.3.smithi137.stderr:+ output='Node Generic SN Model Namespace Usage Format FW Rev 2024-05-21T12:30:10.829 INFO:tasks.workunit.client.3.smithi137.stderr:--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- -------- 2024-05-21T12:30:10.829 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme0n1 /dev/ng0n1 CVFT6233004A400BGN INTEL SSDPEDMD400G4 0x1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0' 2024-05-21T12:30:10.829 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Node Generic SN Model Namespace Usage Format FW Rev 2024-05-21T12:30:10.829 INFO:tasks.workunit.client.3.smithi137.stderr:--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- -------- 2024-05-21T12:30:10.829 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme0n1 /dev/ng0n1 CVFT6233004A400BGN INTEL SSDPEDMD400G4 0x1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0' 2024-05-21T12:30:10.829 INFO:tasks.workunit.client.3.smithi137.stderr:+ grep -q 'Ceph bdev Controller' 2024-05-21T12:30:10.830 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] disconnect_all test passed! 2024-05-21T12:30:10.830 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -eq 0 ']' 2024-05-21T12:30:10.830 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] disconnect_all test passed!' 2024-05-21T12:30:10.830 INFO:tasks.workunit.client.3.smithi137.stderr:+ test_run discovery 2024-05-21T12:30:10.831 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Running test: discovery 2024-05-21T12:30:10.831 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Running test: discovery' 2024-05-21T12:30:10.831 INFO:tasks.workunit.client.3.smithi137.stderr:+ discovery 2024-05-21T12:30:10.831 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme discover -t tcp -a 172.21.15.45 -s 8009 2024-05-21T12:30:10.862 INFO:tasks.workunit.client.3.smithi137.stderr:+ output=' 2024-05-21T12:30:10.862 INFO:tasks.workunit.client.3.smithi137.stderr:Discovery Log Number of Records 9, Generation counter 1 2024-05-21T12:30:10.862 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 0====== 2024-05-21T12:30:10.862 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 0 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.45 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 1====== 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.863 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 1 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.72 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 2====== 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.864 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 2 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.92 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 3====== 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.865 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 3 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode2 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.45 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 4====== 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.866 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 4 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode2 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.72 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 5====== 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 5 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.867 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode2 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.92 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 6====== 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 6 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode3 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.45 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.868 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.869 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 7====== 2024-05-21T12:30:10.869 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.869 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.869 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.869 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.869 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 7 2024-05-21T12:30:10.869 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode3 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.72 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 8====== 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 8 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.870 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode3 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.92 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none' 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:+ expected_discovery_stdout='subtype: nvme subsystem' 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo ' 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:Discovery Log Number of Records 9, Generation counter 1 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 0====== 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 0 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.871 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.45 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 1====== 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 1 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.72 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.872 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 2====== 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 2 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.92 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 3====== 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.873 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 3 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode2 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.45 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 4====== 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.874 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 4 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode2 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.72 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 5====== 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 5 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode2 2024-05-21T12:30:10.875 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.92 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 6====== 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 6 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode3 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.45 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.876 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 7====== 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 7 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode3 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.72 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:=====Discovery Log Entry 8====== 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:trtype: tcp 2024-05-21T12:30:10.877 INFO:tasks.workunit.client.3.smithi137.stderr:adrfam: ipv4 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:subtype: nvme subsystem 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:treq: not required 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:portid: 8 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:trsvcid: 4420 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:subnqn: nqn.2016-06.io.spdk:cnode3 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:traddr: 172.21.15.92 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:eflags: none 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:sectype: none' 2024-05-21T12:30:10.878 INFO:tasks.workunit.client.3.smithi137.stderr:+ grep -q 'subtype: nvme subsystem' 2024-05-21T12:30:10.879 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -eq 0 ']' 2024-05-21T12:30:10.879 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] discovery test passed!' 2024-05-21T12:30:10.879 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] discovery test passed! 2024-05-21T12:30:10.880 INFO:tasks.workunit.client.3.smithi137.stderr:+ test_run connect 2024-05-21T12:30:10.880 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Running test: connect 2024-05-21T12:30:10.880 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Running test: connect' 2024-05-21T12:30:10.881 INFO:tasks.workunit.client.3.smithi137.stderr:+ connect 2024-05-21T12:30:10.881 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo nvme connect -t tcp --traddr 172.21.15.45 -s 4420 -n nqn.2016-06.io.spdk:cnode1 2024-05-21T12:30:10.952 INFO:tasks.workunit.client.3.smithi137.stdout:connecting to device: nvme1 2024-05-21T12:30:10.953 INFO:tasks.workunit.client.3.smithi137.stderr:+ sleep 5 2024-05-21T12:30:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:12 smithi072 ceph-mon[33912]: pgmap v428: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2 op/s 2024-05-21T12:30:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:12 smithi092 ceph-mon[33665]: pgmap v428: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2 op/s 2024-05-21T12:30:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:12 smithi045 ceph-mon[30165]: pgmap v428: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2 op/s 2024-05-21T12:30:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:14 smithi072 ceph-mon[33912]: pgmap v429: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-21T12:30:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:14 smithi092 ceph-mon[33665]: pgmap v429: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-21T12:30:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:14 smithi045 ceph-mon[30165]: pgmap v429: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 255 B/s wr, 4 op/s 2024-05-21T12:30:15.955 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme list --output-format=json 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr:+ output='{ 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: "Devices":[ 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":9, 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n9", 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n9", 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.972 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":6, 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n6", 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n6", 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.973 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":3, 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n3", 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n3", 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.974 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":18, 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n18", 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n18", 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.975 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":15, 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n15", 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n15", 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.976 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.977 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.977 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":12, 2024-05-21T12:30:15.977 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n12", 2024-05-21T12:30:15.977 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n12", 2024-05-21T12:30:15.977 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.977 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.977 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.977 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.978 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme0n1", 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng0n1", 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"8DV101H0", 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-05-21T12:30:15.979 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"CVFT6233004A400BGN", 2024-05-21T12:30:15.980 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":400088457216, 2024-05-21T12:30:15.980 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":781422768, 2024-05-21T12:30:15.980 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":400088457216, 2024-05-21T12:30:15.980 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.980 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:15.980 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:15.980 INFO:tasks.workunit.client.3.smithi137.stderr:}' 2024-05-21T12:30:15.981 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '{ 2024-05-21T12:30:15.981 INFO:tasks.workunit.client.3.smithi137.stderr: "Devices":[ 2024-05-21T12:30:15.981 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.981 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":9, 2024-05-21T12:30:15.981 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n9", 2024-05-21T12:30:15.981 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n9", 2024-05-21T12:30:15.981 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":6, 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n6", 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n6", 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.982 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":3, 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n3", 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n3", 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.983 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":18, 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n18", 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n18", 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.984 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":15, 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n15", 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n15", 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.985 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":12, 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme1n12", 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng1n12", 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:15.986 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:15.987 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:15.987 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.987 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.987 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.987 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:15.987 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme0n1", 2024-05-21T12:30:15.988 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng0n1", 2024-05-21T12:30:15.988 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"8DV101H0", 2024-05-21T12:30:15.988 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-05-21T12:30:15.988 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"CVFT6233004A400BGN", 2024-05-21T12:30:15.988 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":400088457216, 2024-05-21T12:30:15.989 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":781422768, 2024-05-21T12:30:15.989 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":400088457216, 2024-05-21T12:30:15.989 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:15.989 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:15.989 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:15.989 INFO:tasks.workunit.client.3.smithi137.stderr:}' 2024-05-21T12:30:15.989 INFO:tasks.workunit.client.3.smithi137.stderr:+ grep -q 'Ceph bdev Controller' 2024-05-21T12:30:15.989 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] connect test passed! 2024-05-21T12:30:15.990 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -eq 0 ']' 2024-05-21T12:30:15.990 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] connect test passed!' 2024-05-21T12:30:15.990 INFO:tasks.workunit.client.3.smithi137.stderr:+ test_run list_subsys 1 2024-05-21T12:30:15.990 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Running test: list_subsys 2024-05-21T12:30:15.991 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Running test: list_subsys' 2024-05-21T12:30:15.991 INFO:tasks.workunit.client.3.smithi137.stderr:+ list_subsys 1 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr:+ expected_count=1 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme list-subsys --output-format=json 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr:+ output='[ 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: "HostNQN":"nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-0cc47a6bfe00", 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: "HostID":"51abb3f3-aee1-41b2-95cb-1c54f410f09c", 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: "Subsystems":[ 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme-subsys1", 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: "NQN":"nqn.2016-06.io.spdk:cnode1", 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: "IOPolicy":"numa", 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: "Paths":[ 2024-05-21T12:30:15.992 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme1", 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme-subsys0", 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: "NQN":"nqn.2014.08.org.nvmexpress:80868086CVFT6233004A400BGN INTEL SSDPEDMD400G4", 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: "IOPolicy":"numa", 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: "Paths":[ 2024-05-21T12:30:15.993 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme0", 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"pcie", 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"0000:02:00.0", 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr:]' 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr:++ echo '[' '{' '"HostNQN":"nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-0cc47a6bfe00",' '"HostID":"51abb3f3-aee1-41b2-95cb-1c54f410f09c",' '"Subsystems":[' '{' '"Name":"nvme-subsys1",' '"NQN":"nqn.2016-06.io.spdk:cnode1",' '"IOPolicy":"numa",' '"Paths":[' '{' '"Name":"nvme1",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '}' ']' '},' '{' '"Name":"nvme-subsys0",' '"NQN":"nqn.2014.08.org.nvmexpress:80868086CVFT6233004A400BGN' INTEL 'SSDPEDMD400G4",' '"IOPolicy":"numa",' '"Paths":[' '{' '"Name":"nvme0",' '"Transport":"pcie",' '"Address":"0000:02:00.0",' '"State":"live"' '}' ']' '}' ']' '}' ']' 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr:++ grep -o '"tcp"' 2024-05-21T12:30:15.994 INFO:tasks.workunit.client.3.smithi137.stderr:++ wc -l 2024-05-21T12:30:15.995 INFO:tasks.workunit.client.3.smithi137.stderr:+ multipath=1 2024-05-21T12:30:15.995 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 1 -ne 1 ']' 2024-05-21T12:30:15.995 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -eq 0 ']' 2024-05-21T12:30:15.995 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] list_subsys test passed!' 2024-05-21T12:30:15.995 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] list_subsys test passed! 2024-05-21T12:30:15.995 INFO:tasks.workunit.client.3.smithi137.stderr:+ test_run disconnect_all 2024-05-21T12:30:15.995 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Running test: disconnect_all' 2024-05-21T12:30:15.996 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Running test: disconnect_all 2024-05-21T12:30:15.996 INFO:tasks.workunit.client.3.smithi137.stderr:+ disconnect_all 2024-05-21T12:30:15.996 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo nvme disconnect-all 2024-05-21T12:30:16.346 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme list 2024-05-21T12:30:16.360 INFO:tasks.workunit.client.3.smithi137.stderr:+ output='Node Generic SN Model Namespace Usage Format FW Rev 2024-05-21T12:30:16.360 INFO:tasks.workunit.client.3.smithi137.stderr:--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- -------- 2024-05-21T12:30:16.360 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme0n1 /dev/ng0n1 CVFT6233004A400BGN INTEL SSDPEDMD400G4 0x1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0' 2024-05-21T12:30:16.360 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo 'Node Generic SN Model Namespace Usage Format FW Rev 2024-05-21T12:30:16.361 INFO:tasks.workunit.client.3.smithi137.stderr:--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- -------- 2024-05-21T12:30:16.361 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme0n1 /dev/ng0n1 CVFT6233004A400BGN INTEL SSDPEDMD400G4 0x1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0' 2024-05-21T12:30:16.361 INFO:tasks.workunit.client.3.smithi137.stderr:+ grep -q 'Ceph bdev Controller' 2024-05-21T12:30:16.361 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -eq 0 ']' 2024-05-21T12:30:16.361 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] disconnect_all test passed! 2024-05-21T12:30:16.362 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] disconnect_all test passed!' 2024-05-21T12:30:16.362 INFO:tasks.workunit.client.3.smithi137.stderr:+ test_run list_subsys 0 2024-05-21T12:30:16.362 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Running test: list_subsys 2024-05-21T12:30:16.363 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Running test: list_subsys' 2024-05-21T12:30:16.363 INFO:tasks.workunit.client.3.smithi137.stderr:+ list_subsys 0 2024-05-21T12:30:16.363 INFO:tasks.workunit.client.3.smithi137.stderr:+ expected_count=0 2024-05-21T12:30:16.363 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme list-subsys --output-format=json 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr:+ output='[ 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: "HostNQN":"nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-0cc47a6bfe00", 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: "HostID":"51abb3f3-aee1-41b2-95cb-1c54f410f09c", 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: "Subsystems":[ 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme-subsys0", 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: "NQN":"nqn.2014.08.org.nvmexpress:80868086CVFT6233004A400BGN INTEL SSDPEDMD400G4", 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: "IOPolicy":"numa", 2024-05-21T12:30:16.375 INFO:tasks.workunit.client.3.smithi137.stderr: "Paths":[ 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme0", 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"pcie", 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"0000:02:00.0", 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:16.376 INFO:tasks.workunit.client.3.smithi137.stderr:]' 2024-05-21T12:30:16.377 INFO:tasks.workunit.client.3.smithi137.stderr:++ echo '[' '{' '"HostNQN":"nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-0cc47a6bfe00",' '"HostID":"51abb3f3-aee1-41b2-95cb-1c54f410f09c",' '"Subsystems":[' '{' '"Name":"nvme-subsys0",' '"NQN":"nqn.2014.08.org.nvmexpress:80868086CVFT6233004A400BGN' INTEL 'SSDPEDMD400G4",' '"IOPolicy":"numa",' '"Paths":[' '{' '"Name":"nvme0",' '"Transport":"pcie",' '"Address":"0000:02:00.0",' '"State":"live"' '}' ']' '}' ']' '}' ']' 2024-05-21T12:30:16.377 INFO:tasks.workunit.client.3.smithi137.stderr:++ grep -o '"tcp"' 2024-05-21T12:30:16.377 INFO:tasks.workunit.client.3.smithi137.stderr:++ wc -l 2024-05-21T12:30:16.377 INFO:tasks.workunit.client.3.smithi137.stderr:+ multipath=0 2024-05-21T12:30:16.377 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -ne 0 ']' 2024-05-21T12:30:16.377 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -eq 0 ']' 2024-05-21T12:30:16.378 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] list_subsys test passed! 2024-05-21T12:30:16.378 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] list_subsys test passed!' 2024-05-21T12:30:16.378 INFO:tasks.workunit.client.3.smithi137.stderr:+ test_run connect_all 2024-05-21T12:30:16.378 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Running test: connect_all' 2024-05-21T12:30:16.379 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Running test: connect_all 2024-05-21T12:30:16.379 INFO:tasks.workunit.client.3.smithi137.stderr:+ connect_all 2024-05-21T12:30:16.379 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo nvme connect-all --traddr=172.21.15.45 --transport=tcp -l 3600 2024-05-21T12:30:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:16 smithi072 ceph-mon[33912]: pgmap v430: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:30:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:16 smithi092 ceph-mon[33665]: pgmap v430: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:30:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:16 smithi045 ceph-mon[30165]: pgmap v430: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:30:17.178 INFO:tasks.workunit.client.3.smithi137.stderr:+ sleep 5 2024-05-21T12:30:18.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:18 smithi072 ceph-mon[33912]: pgmap v431: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-21T12:30:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:18 smithi092 ceph-mon[33665]: pgmap v431: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-21T12:30:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:18 smithi045 ceph-mon[30165]: pgmap v431: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-21T12:30:20.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:20 smithi072 ceph-mon[33912]: pgmap v432: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:30:20.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:20 smithi092 ceph-mon[33665]: pgmap v432: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:30:20.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:20 smithi045 ceph-mon[30165]: pgmap v432: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-21T12:30:22.179 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme list --output-format=json 2024-05-21T12:30:22.212 INFO:tasks.workunit.client.3.smithi137.stderr:+ output='{ 2024-05-21T12:30:22.212 INFO:tasks.workunit.client.3.smithi137.stderr: "Devices":[ 2024-05-21T12:30:22.212 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":9, 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n9", 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n9", 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.213 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":8, 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n8", 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n8", 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.214 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":7, 2024-05-21T12:30:22.215 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n7", 2024-05-21T12:30:22.215 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n7", 2024-05-21T12:30:22.215 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.215 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.215 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.215 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.215 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.215 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":6, 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n6", 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n6", 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.216 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":5, 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n5", 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n5", 2024-05-21T12:30:22.217 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":4, 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n4", 2024-05-21T12:30:22.218 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n4", 2024-05-21T12:30:22.219 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.219 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.219 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.219 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.219 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.220 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.220 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.220 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.220 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.220 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":3, 2024-05-21T12:30:22.220 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n3", 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n3", 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":20, 2024-05-21T12:30:22.221 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n20", 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n20", 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":2, 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n2", 2024-05-21T12:30:22.222 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n2", 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":19, 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n19", 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n19", 2024-05-21T12:30:22.223 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":18, 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n18", 2024-05-21T12:30:22.224 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n18", 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":17, 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n17", 2024-05-21T12:30:22.225 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n17", 2024-05-21T12:30:22.226 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.226 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.226 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.226 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.226 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.226 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.227 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.227 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.227 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.227 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":16, 2024-05-21T12:30:22.227 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n16", 2024-05-21T12:30:22.227 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n16", 2024-05-21T12:30:22.228 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.228 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.228 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.228 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.228 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.229 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.229 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.229 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.229 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.229 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":15, 2024-05-21T12:30:22.229 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n15", 2024-05-21T12:30:22.229 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n15", 2024-05-21T12:30:22.230 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.230 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.230 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.230 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.230 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.230 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.230 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.230 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":14, 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n14", 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n14", 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.231 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":13, 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n13", 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n13", 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.232 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":12, 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n12", 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n12", 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":11, 2024-05-21T12:30:22.233 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n11", 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n11", 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":10, 2024-05-21T12:30:22.234 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n10", 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n10", 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n1", 2024-05-21T12:30:22.235 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n1", 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":9, 2024-05-21T12:30:22.236 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n9", 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n9", 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":8, 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n8", 2024-05-21T12:30:22.237 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n8", 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.238 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":7, 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n7", 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n7", 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":6, 2024-05-21T12:30:22.239 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n6", 2024-05-21T12:30:22.240 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n6", 2024-05-21T12:30:22.240 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.240 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.240 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.240 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.240 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.240 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":5, 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n5", 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n5", 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.241 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":4, 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n4", 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n4", 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.242 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":3, 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n3", 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n3", 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.243 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":20, 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n20", 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n20", 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.244 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":2, 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n2", 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n2", 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.245 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.246 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.246 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.246 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.246 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.246 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.246 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":19, 2024-05-21T12:30:22.246 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n19", 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n19", 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":18, 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n18", 2024-05-21T12:30:22.247 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n18", 2024-05-21T12:30:22.248 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.248 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.248 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.248 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.248 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.248 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.248 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.248 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.249 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.249 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":17, 2024-05-21T12:30:22.249 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n17", 2024-05-21T12:30:22.249 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n17", 2024-05-21T12:30:22.249 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.249 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.249 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.250 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.250 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.250 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.250 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.250 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.250 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.250 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":16, 2024-05-21T12:30:22.250 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n16", 2024-05-21T12:30:22.251 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n16", 2024-05-21T12:30:22.251 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.251 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.251 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.251 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.251 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.251 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.251 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.252 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.252 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.252 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":15, 2024-05-21T12:30:22.252 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n15", 2024-05-21T12:30:22.252 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n15", 2024-05-21T12:30:22.252 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.252 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.252 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":14, 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n14", 2024-05-21T12:30:22.253 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n14", 2024-05-21T12:30:22.254 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.254 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.254 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.254 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.254 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.254 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.254 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.255 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.255 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.255 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":13, 2024-05-21T12:30:22.255 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n13", 2024-05-21T12:30:22.255 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n13", 2024-05-21T12:30:22.255 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.255 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.255 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":12, 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n12", 2024-05-21T12:30:22.256 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n12", 2024-05-21T12:30:22.257 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.257 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.257 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.257 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.257 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.257 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.257 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.258 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.258 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.258 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":11, 2024-05-21T12:30:22.258 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n11", 2024-05-21T12:30:22.258 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n11", 2024-05-21T12:30:22.258 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.258 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.258 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":10, 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n10", 2024-05-21T12:30:22.259 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n10", 2024-05-21T12:30:22.260 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.260 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.260 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.260 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.260 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.260 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.260 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.260 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n1", 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n1", 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.261 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.262 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.262 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.262 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.262 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.262 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":9, 2024-05-21T12:30:22.262 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n9", 2024-05-21T12:30:22.262 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n9", 2024-05-21T12:30:22.262 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.263 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":8, 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n8", 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n8", 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":7, 2024-05-21T12:30:22.264 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n7", 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n7", 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":6, 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n6", 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n6", 2024-05-21T12:30:22.265 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.266 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.266 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.266 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.266 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.266 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.266 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.266 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.267 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.267 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":5, 2024-05-21T12:30:22.267 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n5", 2024-05-21T12:30:22.267 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n5", 2024-05-21T12:30:22.267 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.267 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.268 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.268 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.268 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.268 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.268 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.268 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.269 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.269 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":4, 2024-05-21T12:30:22.269 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n4", 2024-05-21T12:30:22.269 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n4", 2024-05-21T12:30:22.269 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.269 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.270 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.270 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.270 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.270 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.270 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.270 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.271 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.271 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":3, 2024-05-21T12:30:22.271 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n3", 2024-05-21T12:30:22.271 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n3", 2024-05-21T12:30:22.271 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":20, 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n20", 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n20", 2024-05-21T12:30:22.272 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":2, 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n2", 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n2", 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.273 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":19, 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n19", 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n19", 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.274 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":18, 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n18", 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n18", 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.275 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":17, 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n17", 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n17", 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.276 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":16, 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n16", 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n16", 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.277 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":15, 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n15", 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n15", 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.278 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":14, 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n14", 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n14", 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.279 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":13, 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n13", 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n13", 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":12, 2024-05-21T12:30:22.280 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n12", 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n12", 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":11, 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n11", 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n11", 2024-05-21T12:30:22.281 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":10, 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n10", 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n10", 2024-05-21T12:30:22.282 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n1", 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n1", 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.283 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme0n1", 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng0n1", 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"8DV101H0", 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"CVFT6233004A400BGN", 2024-05-21T12:30:22.284 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":400088457216, 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":781422768, 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":400088457216, 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr:}' 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr:+ grep -q 'Ceph bdev Controller' 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '{ 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: "Devices":[ 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":9, 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n9", 2024-05-21T12:30:22.285 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n9", 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":8, 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n8", 2024-05-21T12:30:22.286 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n8", 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":7, 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n7", 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n7", 2024-05-21T12:30:22.287 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":6, 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n6", 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n6", 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.288 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":5, 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n5", 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n5", 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.289 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":4, 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n4", 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n4", 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.290 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":3, 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n3", 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n3", 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.291 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":20, 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n20", 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n20", 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.292 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":2, 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n2", 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n2", 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.293 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":19, 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n19", 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n19", 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.294 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":18, 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n18", 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n18", 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.295 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":17, 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n17", 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n17", 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.296 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":16, 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n16", 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n16", 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.297 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":15, 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n15", 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n15", 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.298 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":14, 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n14", 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n14", 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.299 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":13, 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n13", 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n13", 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":12, 2024-05-21T12:30:22.300 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n12", 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n12", 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":11, 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n11", 2024-05-21T12:30:22.301 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n11", 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":10, 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n10", 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n10", 2024-05-21T12:30:22.302 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme5n1", 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng5n1", 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.303 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph77937083268835", 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":9, 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n9", 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n9", 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.304 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":8, 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n8", 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n8", 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.305 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":7, 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n7", 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n7", 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.306 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":6, 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n6", 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n6", 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.307 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":5, 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n5", 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n5", 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.308 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":4, 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n4", 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n4", 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.309 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":3, 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n3", 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n3", 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.310 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":20, 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n20", 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n20", 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.311 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":2, 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n2", 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n2", 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.312 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":19, 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n19", 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n19", 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.313 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":18, 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n18", 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n18", 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":17, 2024-05-21T12:30:22.314 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n17", 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n17", 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":16, 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n16", 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n16", 2024-05-21T12:30:22.315 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":15, 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n15", 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n15", 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.316 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":14, 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n14", 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n14", 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.317 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":13, 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n13", 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n13", 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.318 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":12, 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n12", 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n12", 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.319 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":11, 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n11", 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n11", 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.320 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":10, 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n10", 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n10", 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:22.321 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme2n1", 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng2n1", 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph40086972576734", 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":9, 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n9", 2024-05-21T12:30:22.322 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n9", 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":8, 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n8", 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n8", 2024-05-21T12:30:22.323 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":7, 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n7", 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n7", 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.324 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":6, 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n6", 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n6", 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.325 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":5, 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n5", 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n5", 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.326 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":4, 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n4", 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n4", 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.327 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":3, 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n3", 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n3", 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.328 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":20, 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n20", 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n20", 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.329 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":2, 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n2", 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n2", 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.330 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":19, 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n19", 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n19", 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.331 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":18, 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n18", 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n18", 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":17, 2024-05-21T12:30:22.332 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n17", 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n17", 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":16, 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n16", 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n16", 2024-05-21T12:30:22.333 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":15, 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n15", 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n15", 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.334 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":14, 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n14", 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n14", 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.335 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":13, 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n13", 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n13", 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.336 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":12, 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n12", 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n12", 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.337 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":11, 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n11", 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n11", 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.338 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":10, 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n10", 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n10", 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.339 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme8n1", 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng8n1", 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"24.01", 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"Ceph bdev Controller", 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"Ceph15464245260738", 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":8589934592, 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":16777216, 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":8589934592, 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "NameSpace":1, 2024-05-21T12:30:22.340 INFO:tasks.workunit.client.3.smithi137.stderr: "DevicePath":"/dev/nvme0n1", 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: "GenericPath":"/dev/ng0n1", 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: "Firmware":"8DV101H0", 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: "SerialNumber":"CVFT6233004A400BGN", 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: "UsedBytes":400088457216, 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: "MaximumLBA":781422768, 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: "PhysicalSize":400088457216, 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: "SectorSize":512 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:22.341 INFO:tasks.workunit.client.3.smithi137.stderr:}' 2024-05-21T12:30:22.342 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] connect_all test passed! 2024-05-21T12:30:22.342 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -eq 0 ']' 2024-05-21T12:30:22.343 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] connect_all test passed!' 2024-05-21T12:30:22.343 INFO:tasks.workunit.client.3.smithi137.stderr:++ echo 172.21.15.45,172.21.15.72,172.21.15.92 2024-05-21T12:30:22.343 INFO:tasks.workunit.client.3.smithi137.stderr:++ tr -cd , 2024-05-21T12:30:22.343 INFO:tasks.workunit.client.3.smithi137.stderr:++ wc -c 2024-05-21T12:30:22.343 INFO:tasks.workunit.client.3.smithi137.stderr:+ gateways_count=3 2024-05-21T12:30:22.343 INFO:tasks.workunit.client.3.smithi137.stderr:+ multipath_count=9 2024-05-21T12:30:22.343 INFO:tasks.workunit.client.3.smithi137.stderr:+ test_run list_subsys 9 2024-05-21T12:30:22.343 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] Running test: list_subsys 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] Running test: list_subsys' 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr:+ list_subsys 9 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr:+ expected_count=9 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme list-subsys --output-format=json 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr:+ output='[ 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr: "HostNQN":"nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-0cc47a6bfe00", 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr: "HostID":"51abb3f3-aee1-41b2-95cb-1c54f410f09c", 2024-05-21T12:30:22.345 INFO:tasks.workunit.client.3.smithi137.stderr: "Subsystems":[ 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme-subsys5", 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "NQN":"nqn.2016-06.io.spdk:cnode2", 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "IOPolicy":"numa", 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "Paths":[ 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme7", 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.346 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme6", 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme5", 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.347 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme-subsys2", 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "NQN":"nqn.2016-06.io.spdk:cnode1", 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "IOPolicy":"numa", 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "Paths":[ 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme4", 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme3", 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.348 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme2", 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme-subsys8", 2024-05-21T12:30:22.349 INFO:tasks.workunit.client.3.smithi137.stderr: "NQN":"nqn.2016-06.io.spdk:cnode3", 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "IOPolicy":"numa", 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "Paths":[ 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme9", 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme8", 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.350 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme10", 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"tcp", 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137", 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: }, 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme-subsys0", 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: "NQN":"nqn.2014.08.org.nvmexpress:80868086CVFT6233004A400BGN INTEL SSDPEDMD400G4", 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: "IOPolicy":"numa", 2024-05-21T12:30:22.351 INFO:tasks.workunit.client.3.smithi137.stderr: "Paths":[ 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: { 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: "Name":"nvme0", 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: "Transport":"pcie", 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: "Address":"0000:02:00.0", 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: "State":"live" 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: ] 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr: } 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr:]' 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr:++ echo '[' '{' '"HostNQN":"nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-0cc47a6bfe00",' '"HostID":"51abb3f3-aee1-41b2-95cb-1c54f410f09c",' '"Subsystems":[' '{' '"Name":"nvme-subsys5",' '"NQN":"nqn.2016-06.io.spdk:cnode2",' '"IOPolicy":"numa",' '"Paths":[' '{' '"Name":"nvme7",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '},' '{' '"Name":"nvme6",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '},' '{' '"Name":"nvme5",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '}' ']' '},' '{' '"Name":"nvme-subsys2",' '"NQN":"nqn.2016-06.io.spdk:cnode1",' '"IOPolicy":"numa",' '"Paths":[' '{' '"Name":"nvme4",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '},' '{' '"Name":"nvme3",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '},' '{' '"Name":"nvme2",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '}' ']' '},' '{' '"Name":"nvme-subsys8",' '"NQN":"nqn.2016-06.io.spdk:cnode3",' '"IOPolicy":"numa",' '"Paths":[' '{' '"Name":"nvme9",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '},' '{' '"Name":"nvme8",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '},' '{' '"Name":"nvme10",' '"Transport":"tcp",' '"Address":"traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137",' '"State":"live"' '}' ']' '},' '{' '"Name":"nvme-subsys0",' '"NQN":"nqn.2014.08.org.nvmexpress:80868086CVFT6233004A400BGN' INTEL 'SSDPEDMD400G4",' '"IOPolicy":"numa",' '"Paths":[' '{' '"Name":"nvme0",' '"Transport":"pcie",' '"Address":"0000:02:00.0",' '"State":"live"' '}' ']' '}' ']' '}' ']' 2024-05-21T12:30:22.352 INFO:tasks.workunit.client.3.smithi137.stderr:++ grep -o '"tcp"' 2024-05-21T12:30:22.353 INFO:tasks.workunit.client.3.smithi137.stderr:++ wc -l 2024-05-21T12:30:22.353 INFO:tasks.workunit.client.3.smithi137.stderr:+ multipath=9 2024-05-21T12:30:22.353 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 9 -ne 9 ']' 2024-05-21T12:30:22.353 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' 0 -eq 0 ']' 2024-05-21T12:30:22.353 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] list_subsys test passed! 2024-05-21T12:30:22.353 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] list_subsys test passed!' 2024-05-21T12:30:22.354 INFO:tasks.workunit.client.3.smithi137.stdout:-------------Test Summary------------- 2024-05-21T12:30:22.354 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '-------------Test Summary-------------' 2024-05-21T12:30:22.354 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof] All nvmeof basic tests passed! 2024-05-21T12:30:22.355 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof] All nvmeof basic tests passed!' 2024-05-21T12:30:22.355 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-21T12:30:22.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.3/client.3/tmp 2024-05-21T12:30:22.424 INFO:tasks.workunit:Stopping ['rbd/nvmeof_setup_subsystem.sh', 'rbd/nvmeof_basic_tests.sh'] on client.3... 2024-05-21T12:30:22.424 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.3 /home/ubuntu/cephtest/clone.client.3 2024-05-21T12:30:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:22 smithi072 ceph-mon[33912]: pgmap v433: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-21T12:30:22.744 DEBUG:teuthology.parallel:result is None 2024-05-21T12:30:22.744 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.3/client.3 2024-05-21T12:30:22.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:22 smithi092 ceph-mon[33665]: pgmap v433: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-21T12:30:22.770 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.3/client.3 2024-05-21T12:30:22.770 DEBUG:teuthology.orchestra.run.smithi137:> rmdir -- /home/ubuntu/cephtest/mnt.3 2024-05-21T12:30:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:22 smithi045 ceph-mon[30165]: pgmap v433: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-21T12:30:22.840 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.3/client.3 2024-05-21T12:30:22.840 INFO:teuthology.run_tasks:Running task nvmeof.thrash... 2024-05-21T12:30:22.855 DEBUG:teuthology.orchestra.run.smithi137:> sudo nvme list --output-format=json | jq -r '.Devices | sort_by(.NameSpace) | .[] | select(.ModelNumber == "Ceph bdev Controller") | .DevicePath' 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n1 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n1 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n1 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n2 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n2 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n2 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n3 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n3 2024-05-21T12:30:22.919 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n3 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n4 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n4 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n4 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n5 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n5 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n5 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n6 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n6 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n6 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n7 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n7 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n7 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n8 2024-05-21T12:30:22.920 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n8 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n8 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n9 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n9 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n9 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n10 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n10 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n10 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n11 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n11 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n11 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n12 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n12 2024-05-21T12:30:22.921 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n12 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n13 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n13 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n13 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n14 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n14 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n14 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n15 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n15 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n15 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n16 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n16 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n16 2024-05-21T12:30:22.922 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n17 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n17 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n17 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n18 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n18 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n18 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n19 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n19 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n19 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme5n20 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme2n20 2024-05-21T12:30:22.923 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/nvme8n20 2024-05-21T12:30:22.924 INFO:teuthology.run_tasks:Running task workunit... 2024-05-21T12:30:22.932 INFO:tasks.workunit:Pulling workunits from ref 68fa3266016634745d991c9967c479c757966afe 2024-05-21T12:30:22.933 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-21T12:30:22.933 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.3 2024-05-21T12:30:22.933 INFO:tasks.nvmeof.[nvmeof.thrasher]:start thrashing 2024-05-21T12:30:22.933 INFO:tasks.nvmeof.[nvmeof.thrasher]:seed: 1716294622, , max thrash delay: 90, min thrash delay: 60 max revive delay: 130, min revive delay: 100 daemons: 3 2024-05-21T12:30:22.933 INFO:tasks.nvmeof.[nvmeof.thrasher]:kill nvmeof.a 2024-05-21T12:30:22.934 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Stopping nvmeof.nvmeof.a... 2024-05-21T12:30:22.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:30:22.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:30:22.974 INFO:teuthology.orchestra.run.smithi137.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-21T12:30:22.975 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -- /home/ubuntu/cephtest/mnt.3 2024-05-21T12:30:23.030 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.3 2024-05-21T12:30:23.030 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.3 && mkdir -- client.3 2024-05-21T12:30:23.085 INFO:tasks.workunit:timeout=30m 2024-05-21T12:30:23.085 INFO:tasks.workunit:cleanup=True 2024-05-21T12:30:23.085 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.3 && git clone https://github.com/VallariAg/ceph /home/ubuntu/cephtest/clone.client.3 && cd /home/ubuntu/cephtest/clone.client.3 && git checkout 68fa3266016634745d991c9967c479c757966afe 2024-05-21T12:30:23.140 INFO:tasks.workunit.client.3.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.3'... 2024-05-21T12:30:23.249 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:22 smithi045 systemd[1]: Stopping Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:30:23.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:30:23] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log.gz 2024-05-21T12:30:24.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:24 smithi072 ceph-mon[33912]: pgmap v434: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-21T12:30:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:24 smithi092 ceph-mon[33665]: pgmap v434: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-21T12:30:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:24 smithi045 ceph-mon[30165]: pgmap v434: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-21T12:30:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:26 smithi072 ceph-mon[33912]: pgmap v435: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-21T12:30:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:26 smithi092 ceph-mon[33665]: pgmap v435: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-21T12:30:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:26 smithi045 ceph-mon[30165]: pgmap v435: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-21T12:30:28.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:28 smithi072 ceph-mon[33912]: pgmap v436: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-21T12:30:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:28 smithi092 ceph-mon[33665]: pgmap v436: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-21T12:30:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:28 smithi045 ceph-mon[30165]: pgmap v436: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-21T12:30:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:30 smithi072 ceph-mon[33912]: pgmap v437: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-21T12:30:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:30 smithi092 ceph-mon[33665]: pgmap v437: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-21T12:30:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:30 smithi045 ceph-mon[30165]: pgmap v437: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-05-21T12:30:32.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:32 smithi072 ceph-mon[33912]: pgmap v438: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:30:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:32 smithi092 ceph-mon[33665]: pgmap v438: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:30:32.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:32 smithi045 ceph-mon[30165]: pgmap v438: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-21T12:30:33.556 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:33 smithi045 bash[72391]: time="2024-05-21T12:30:33Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a in 10 seconds, resorting to SIGKILL" 2024-05-21T12:30:33.556 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:33 smithi045 podman[72391]: 2024-05-21 12:30:33.264117934 +0000 UTC m=+10.123597432 container died 4d34dcd5cab0f2eb889b7fac4e0f0a4afe77f19e67be6a91c8b786a42e67b4d5 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.version=24.01, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.branch=undefined, io.ceph.summary=Ceph NVMe over Fabrics Gateway, build-date=2024-05-19 17:57:19 UTC, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.git.repo=https://github.com/ceph/spdk.git, com.redhat.component=ubi9-container, vendor=Red Hat, Inc., io.ceph.git.branch=tags/1.2.9, io.ceph.version=1.2.9, release=, io.ceph.component=ceph-nvmeof, url=https://github.com/ceph/ceph-nvmeof, io.ceph.nvmeof=None, io.spdk.build.ceph-release=19.0.0, io.spdk.build.pkgdep-args=--rbd, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.target-arch=, org.centos.version=9.0-21.el9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, distribution-scope=public, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.makeflags=, vcs-type=git, architecture=x86_64, io.openshift.wants=ceph,rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.buildah.version=1.29.0, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, version=1.2.9, io.ceph.url=https://github.com/ceph/ceph-nvmeof, summary=Ceph NVMe over Fabrics Gateway, io.ceph.maintainer=Ceph Developers ) 2024-05-21T12:30:33.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:33 smithi045 podman[72391]: 2024-05-21 12:30:33.696774781 +0000 UTC m=+10.556254260 container cleanup 4d34dcd5cab0f2eb889b7fac4e0f0a4afe77f19e67be6a91c8b786a42e67b4d5 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.ceph-release=19.0.0, io.spdk.build.target-arch=, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.buildah.version=1.29.0, org.centos.version=9.0-21.el9, release=, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, version=1.2.9, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, com.redhat.component=ubi9-container, name=ceph-nvmeof, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, summary=Ceph NVMe over Fabrics Gateway, io.ceph.version=1.2.9, io.spdk.build.pkgdep-args=--rbd, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.spdk.build.makeflags=, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vendor=Red Hat, Inc., io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, url=https://github.com/ceph/ceph-nvmeof, io.ceph.summary=Ceph NVMe over Fabrics Gateway, distribution-scope=public, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.branch=undefined, io.openshift.wants=ceph,rbd, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., architecture=x86_64, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.component=ceph-nvmeof, io.ceph.maintainer=Ceph Developers , vcs-type=git, build-date=2024-05-19 17:57:19 UTC, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.nvmeof=None, io.spdk.version=24.01, io.ceph.git.branch=tags/1.2.9, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-05-21T12:30:33.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:30:33 smithi045 bash[72391]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a 2024-05-21T12:30:34.284 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service' 2024-05-21T12:30:34.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:30:34.353 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Stopped nvmeof.nvmeof.a 2024-05-21T12:30:34.353 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.b with skip (0.7146776957229422) > weight (0.3333333333333333) 2024-05-21T12:30:34.353 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.c with skip (0.39944103026258415) > weight (0.3333333333333333) 2024-05-21T12:30:34.354 INFO:tasks.nvmeof.[nvmeof.thrasher]:waiting for 122 secs before reviving 2024-05-21T12:30:34.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:34 smithi072 ceph-mon[33912]: pgmap v439: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:30:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:34 smithi092 ceph-mon[33665]: pgmap v439: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:30:34.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:34 smithi045 ceph-mon[30165]: pgmap v439: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-21T12:30:36.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:36 smithi072 ceph-mon[33912]: pgmap v440: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:30:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:36 smithi092 ceph-mon[33665]: pgmap v440: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:30:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:36 smithi045 ceph-mon[30165]: pgmap v440: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:30:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:38 smithi072 ceph-mon[33912]: pgmap v441: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:30:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:38 smithi092 ceph-mon[33665]: pgmap v441: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:30:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:38 smithi045 ceph-mon[30165]: pgmap v441: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:30:40.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:40 smithi072 ceph-mon[33912]: pgmap v442: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1 op/s 2024-05-21T12:30:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:40 smithi092 ceph-mon[33665]: pgmap v442: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1 op/s 2024-05-21T12:30:40.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:40 smithi045 ceph-mon[30165]: pgmap v442: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1 op/s 2024-05-21T12:30:42.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:42 smithi072 ceph-mon[33912]: pgmap v443: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1 op/s 2024-05-21T12:30:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:42 smithi092 ceph-mon[33665]: pgmap v443: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1 op/s 2024-05-21T12:30:42.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:42 smithi045 ceph-mon[30165]: pgmap v443: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1 op/s 2024-05-21T12:30:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:30:43] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:30:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:30:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:30:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:30:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:30:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:30:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: ], , client address: IPv4 172.21.15.72:37978 2024-05-21T12:30:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:30:43.008150] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000259 seconds 2024-05-21T12:30:44.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:43 smithi092 ceph-mon[33665]: osdmap e57: 8 total, 8 up, 8 in 2024-05-21T12:30:44.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:43 smithi092 ceph-mon[33665]: pgmap v445: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:43 smithi045 ceph-mon[30165]: osdmap e57: 8 total, 8 up, 8 in 2024-05-21T12:30:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:43 smithi045 ceph-mon[30165]: pgmap v445: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-mon[33912]: osdmap e57: 8 total, 8 up, 8 in 2024-05-21T12:30:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:43 smithi072 ceph-mon[33912]: pgmap v445: 33 pgs: 33 active+clean; 609 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:30:45] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:30:45.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:30:45.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:30:45.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:30:45.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:30:45.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:30:45.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:30:45.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: ], , client address: IPv4 172.21.15.72:57980 2024-05-21T12:30:45.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:30:45.012275] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000267 seconds 2024-05-21T12:30:46.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:46 smithi045 ceph-mon[30165]: pgmap v446: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:46.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:46 smithi072 ceph-mon[33912]: pgmap v446: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:46 smithi092 ceph-mon[33665]: pgmap v446: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:47.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:47 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:30:47.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:47 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:30:47.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:47 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:30:48.316 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:30:48.317 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:30:48.317 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:30:48.317 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:48 smithi072 ceph-mon[33912]: pgmap v447: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:48.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:30:48.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:30:48.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:30:48.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:48 smithi045 ceph-mon[30165]: pgmap v447: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:30:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:30:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:30:48.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:48 smithi092 ceph-mon[33665]: pgmap v447: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:50.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:50 smithi072 ceph-mon[33912]: pgmap v448: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:50 smithi092 ceph-mon[33665]: pgmap v448: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:50 smithi045 ceph-mon[30165]: pgmap v448: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:52.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:52 smithi045 ceph-mon[30165]: pgmap v449: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:52.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:52 smithi072 ceph-mon[33912]: pgmap v449: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:52 smithi092 ceph-mon[33665]: pgmap v449: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:54.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:54 smithi072 ceph-mon[33912]: pgmap v450: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:54 smithi092 ceph-mon[33665]: pgmap v450: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:54.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:54 smithi045 ceph-mon[30165]: pgmap v450: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:30:56.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:56 smithi072 ceph-mon[33912]: pgmap v451: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:30:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:56 smithi092 ceph-mon[33665]: pgmap v451: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:30:56.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:56 smithi045 ceph-mon[30165]: pgmap v451: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:30:58.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:30:58 smithi072 ceph-mon[33912]: pgmap v452: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:30:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:30:58 smithi092 ceph-mon[33665]: pgmap v452: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:30:58.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:30:58 smithi045 ceph-mon[30165]: pgmap v452: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:00.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:00 smithi072 ceph-mon[33912]: pgmap v453: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:00 smithi092 ceph-mon[33665]: pgmap v453: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:00 smithi045 ceph-mon[30165]: pgmap v453: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:02.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:02 smithi072 ceph-mon[33912]: pgmap v454: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:02 smithi092 ceph-mon[33665]: pgmap v454: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:02.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:02 smithi045 ceph-mon[30165]: pgmap v454: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:04.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:04 smithi072 ceph-mon[33912]: pgmap v455: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:04 smithi092 ceph-mon[33665]: pgmap v455: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:04 smithi045 ceph-mon[30165]: pgmap v455: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:06 smithi072 ceph-mon[33912]: pgmap v456: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:06 smithi092 ceph-mon[33665]: pgmap v456: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:06 smithi045 ceph-mon[30165]: pgmap v456: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:08.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:08 smithi072 ceph-mon[33912]: pgmap v457: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:08.764 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:08 smithi092 ceph-mon[33665]: pgmap v457: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:08 smithi045 ceph-mon[30165]: pgmap v457: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:10 smithi072 ceph-mon[33912]: pgmap v458: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:10.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:10 smithi092 ceph-mon[33665]: pgmap v458: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:10 smithi045 ceph-mon[30165]: pgmap v458: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:12 smithi072 ceph-mon[33912]: pgmap v459: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:12 smithi092 ceph-mon[33665]: pgmap v459: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:12 smithi045 ceph-mon[30165]: pgmap v459: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:14 smithi072 ceph-mon[33912]: pgmap v460: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:14 smithi092 ceph-mon[33665]: pgmap v460: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:14 smithi045 ceph-mon[30165]: pgmap v460: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:16 smithi072 ceph-mon[33912]: pgmap v461: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:16 smithi092 ceph-mon[33665]: pgmap v461: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:16 smithi045 ceph-mon[30165]: pgmap v461: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:18.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:18 smithi072 ceph-mon[33912]: pgmap v462: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:18 smithi092 ceph-mon[33665]: pgmap v462: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:18 smithi045 ceph-mon[30165]: pgmap v462: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:20.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:20 smithi072 ceph-mon[33912]: pgmap v463: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:20.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:20 smithi092 ceph-mon[33665]: pgmap v463: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:20.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:20 smithi045 ceph-mon[30165]: pgmap v463: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:22 smithi072 ceph-mon[33912]: pgmap v464: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:22 smithi092 ceph-mon[33665]: pgmap v464: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:22 smithi045 ceph-mon[30165]: pgmap v464: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:24 smithi092 ceph-mon[33665]: pgmap v465: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:24 smithi045 ceph-mon[30165]: pgmap v465: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:24.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:24 smithi072 ceph-mon[33912]: pgmap v465: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:26 smithi092 ceph-mon[33665]: pgmap v466: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:26.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:26 smithi045 ceph-mon[30165]: pgmap v466: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:26.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:26 smithi072 ceph-mon[33912]: pgmap v466: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:28 smithi092 ceph-mon[33665]: pgmap v467: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:28 smithi045 ceph-mon[30165]: pgmap v467: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:28.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:28 smithi072 ceph-mon[33912]: pgmap v467: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:30 smithi092 ceph-mon[33665]: pgmap v468: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:30.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:30 smithi045 ceph-mon[30165]: pgmap v468: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:30.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:30 smithi072 ceph-mon[33912]: pgmap v468: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:32 smithi092 ceph-mon[33665]: pgmap v469: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:32.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:32 smithi045 ceph-mon[30165]: pgmap v469: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:32.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:32 smithi072 ceph-mon[33912]: pgmap v469: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:34 smithi092 ceph-mon[33665]: pgmap v470: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:34.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:34 smithi045 ceph-mon[30165]: pgmap v470: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:34.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:34 smithi072 ceph-mon[33912]: pgmap v470: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:36 smithi092 ceph-mon[33665]: pgmap v471: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:36 smithi045 ceph-mon[30165]: pgmap v471: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:36.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:36 smithi072 ceph-mon[33912]: pgmap v471: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:38 smithi092 ceph-mon[33665]: pgmap v472: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:38 smithi045 ceph-mon[30165]: pgmap v472: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:38.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:38 smithi072 ceph-mon[33912]: pgmap v472: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:40 smithi092 ceph-mon[33665]: pgmap v473: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:40.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:40 smithi045 ceph-mon[30165]: pgmap v473: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:40.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:40 smithi072 ceph-mon[33912]: pgmap v473: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:42 smithi092 ceph-mon[33665]: pgmap v474: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:42.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:42 smithi045 ceph-mon[30165]: pgmap v474: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:42.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:42 smithi072 ceph-mon[33912]: pgmap v474: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:44 smithi092 ceph-mon[33665]: pgmap v475: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:44.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:44 smithi045 ceph-mon[30165]: pgmap v475: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:44.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:44 smithi072 ceph-mon[33912]: pgmap v475: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:46 smithi092 ceph-mon[33665]: pgmap v476: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:46.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:46 smithi045 ceph-mon[30165]: pgmap v476: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:46.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:46 smithi072 ceph-mon[33912]: pgmap v476: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:47.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:47 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:31:47.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:47 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:31:47.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:47 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:31:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:48 smithi092 ceph-mon[33665]: pgmap v477: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:31:48.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:31:48.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:31:48.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:31:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:48 smithi045 ceph-mon[30165]: pgmap v477: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:48 smithi072 ceph-mon[33912]: pgmap v477: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.817 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.818 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-05-21T12:31:48.818 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:31:48.818 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:31:48.818 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:31:48.818 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:31:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:49 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-21T12:31:49.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:49 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-21T12:31:49.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:49 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-21T12:31:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:50 smithi092 ceph-mon[33665]: pgmap v478: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:50 smithi045 ceph-mon[30165]: pgmap v478: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:50 smithi072 ceph-mon[33912]: pgmap v478: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:52 smithi092 ceph-mon[33665]: pgmap v479: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:52.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:52 smithi045 ceph-mon[30165]: pgmap v479: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:52.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:52 smithi072 ceph-mon[33912]: pgmap v479: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:54 smithi092 ceph-mon[33665]: pgmap v480: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:54.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:54 smithi045 ceph-mon[30165]: pgmap v480: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:54.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:54 smithi072 ceph-mon[33912]: pgmap v480: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:56.350 INFO:tasks.workunit.client.3.smithi137.stderr:Note: switching to '68fa3266016634745d991c9967c479c757966afe'. 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr: git switch -c 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:31:56.351 INFO:tasks.workunit.client.3.smithi137.stderr:Or undo this operation with: 2024-05-21T12:31:56.352 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:31:56.352 INFO:tasks.workunit.client.3.smithi137.stderr: git switch - 2024-05-21T12:31:56.352 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:31:56.352 INFO:tasks.workunit.client.3.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-21T12:31:56.352 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:31:56.352 INFO:tasks.workunit.client.3.smithi137.stderr:HEAD is now at 68fa3266016 restructure into "basic" and "thrash" subsuites 2024-05-21T12:31:56.356 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.3/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.3 2024-05-21T12:31:56.411 INFO:tasks.workunit.client.3.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-21T12:31:56.413 INFO:tasks.workunit.client.3.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.3/qa/workunits/direct_io' 2024-05-21T12:31:56.414 INFO:tasks.workunit.client.3.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-21T12:31:56.462 INFO:tasks.workunit.client.3.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-21T12:31:56.511 INFO:tasks.workunit.client.3.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-21T12:31:56.550 INFO:tasks.workunit.client.3.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.3/qa/workunits/direct_io' 2024-05-21T12:31:56.552 INFO:tasks.workunit.client.3.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.3/qa/workunits/fs' 2024-05-21T12:31:56.552 INFO:tasks.workunit.client.3.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-21T12:31:56.591 INFO:tasks.workunit.client.3.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.3/qa/workunits/fs' 2024-05-21T12:31:56.594 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-21T12:31:56.595 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.3 of=/dev/stdout 2024-05-21T12:31:56.652 INFO:tasks.workunit:Running workunits matching rbd/nvmeof_fio_test.sh on client.3... 2024-05-21T12:31:56.653 INFO:tasks.workunit:Running workunit rbd/nvmeof_fio_test.sh... 2024-05-21T12:31:56.653 DEBUG:teuthology.orchestra.run.smithi137:workunit test rbd/nvmeof_fio_test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && cd -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68fa3266016634745d991c9967c479c757966afe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="3" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.3 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.3 CEPH_MNT=/home/ubuntu/cephtest/mnt.3 IOSTAT_INTERVAL=10 RBD_POOL=mypool RUNTIME=600 timeout 30m /home/ubuntu/cephtest/clone.client.3/qa/workunits/rbd/nvmeof_fio_test.sh --rbd_iostat 2024-05-21T12:31:56.709 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo yum -y install fio 2024-05-21T12:31:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:56 smithi092 ceph-mon[33665]: pgmap v481: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:56.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:56 smithi045 ceph-mon[30165]: pgmap v481: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:56.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:56 smithi072 ceph-mon[33912]: pgmap v481: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:31:57.225 INFO:tasks.workunit.client.3.smithi137.stdout:Last metadata expiration check: 0:19:22 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:31:57.348 INFO:tasks.workunit.client.3.smithi137.stdout:Dependencies resolved. 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout:================================================================================ 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout: Package Architecture Version Repository Size 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout:================================================================================ 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout:Installing: 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout: fio x86_64 3.35-1.el9 appstream 6.2 M 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout:Installing dependencies: 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout: libnbd x86_64 1.20.0-1.el9 appstream 170 k 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout:Installing weak dependencies: 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-http x86_64 3.35-1.el9 appstream 18 k 2024-05-21T12:31:57.349 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-libaio x86_64 3.35-1.el9 appstream 16 k 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-nbd x86_64 3.35-1.el9 appstream 15 k 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-rados x86_64 3.35-1.el9 appstream 16 k 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-rbd x86_64 3.35-1.el9 appstream 16 k 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-rdma x86_64 3.35-1.el9 appstream 21 k 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout:Transaction Summary 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout:================================================================================ 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout:Install 8 Packages 2024-05-21T12:31:57.350 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:31:57.351 INFO:tasks.workunit.client.3.smithi137.stdout:Total download size: 6.5 M 2024-05-21T12:31:57.351 INFO:tasks.workunit.client.3.smithi137.stdout:Installed size: 9.1 M 2024-05-21T12:31:57.351 INFO:tasks.workunit.client.3.smithi137.stdout:Downloading Packages: 2024-05-21T12:31:57.799 INFO:tasks.workunit.client.3.smithi137.stdout:(1/8): fio-engine-http-3.35-1.el9.x86_64.rpm 101 kB/s | 18 kB 00:00 2024-05-21T12:31:57.824 INFO:tasks.workunit.client.3.smithi137.stdout:(2/8): fio-engine-libaio-3.35-1.el9.x86_64.rpm 77 kB/s | 16 kB 00:00 2024-05-21T12:31:57.849 INFO:tasks.workunit.client.3.smithi137.stdout:(3/8): fio-engine-nbd-3.35-1.el9.x86_64.rpm 296 kB/s | 15 kB 00:00 2024-05-21T12:31:57.874 INFO:tasks.workunit.client.3.smithi137.stdout:(4/8): fio-engine-rados-3.35-1.el9.x86_64.rpm 313 kB/s | 16 kB 00:00 2024-05-21T12:31:57.900 INFO:tasks.workunit.client.3.smithi137.stdout:(5/8): fio-engine-rbd-3.35-1.el9.x86_64.rpm 323 kB/s | 16 kB 00:00 2024-05-21T12:31:57.925 INFO:tasks.workunit.client.3.smithi137.stdout:(6/8): fio-engine-rdma-3.35-1.el9.x86_64.rpm 410 kB/s | 21 kB 00:00 2024-05-21T12:31:58.017 INFO:tasks.workunit.client.3.smithi137.stdout:(7/8): libnbd-1.20.0-1.el9.x86_64.rpm 1.4 MB/s | 170 kB 00:00 2024-05-21T12:31:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:31:58 smithi092 ceph-mon[33665]: pgmap v482: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:58.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:31:58 smithi045 ceph-mon[30165]: pgmap v482: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:58.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:31:58 smithi072 ceph-mon[33912]: pgmap v482: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:31:59.709 INFO:tasks.workunit.client.3.smithi137.stdout:(8/8): fio-3.35-1.el9.x86_64.rpm 3.0 MB/s | 6.2 MB 00:02 2024-05-21T12:31:59.709 INFO:tasks.workunit.client.3.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-21T12:31:59.709 INFO:tasks.workunit.client.3.smithi137.stdout:Total 2.8 MB/s | 6.5 MB 00:02 2024-05-21T12:31:59.838 INFO:tasks.workunit.client.3.smithi137.stdout:Running transaction check 2024-05-21T12:31:59.850 INFO:tasks.workunit.client.3.smithi137.stdout:Transaction check succeeded. 2024-05-21T12:31:59.851 INFO:tasks.workunit.client.3.smithi137.stdout:Running transaction test 2024-05-21T12:31:59.978 INFO:tasks.workunit.client.3.smithi137.stdout:Transaction test succeeded. 2024-05-21T12:31:59.979 INFO:tasks.workunit.client.3.smithi137.stdout:Running transaction 2024-05-21T12:32:00.368 INFO:tasks.workunit.client.3.smithi137.stdout: Preparing : 1/1 2024-05-21T12:32:00.436 INFO:tasks.workunit.client.3.smithi137.stdout: Installing : libnbd-1.20.0-1.el9.x86_64 1/8 2024-05-21T12:32:00.461 INFO:tasks.workunit.client.3.smithi137.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 2/8 2024-05-21T12:32:00.494 INFO:tasks.workunit.client.3.smithi137.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 3/8 2024-05-21T12:32:00.520 INFO:tasks.workunit.client.3.smithi137.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 4/8 2024-05-21T12:32:00.554 INFO:tasks.workunit.client.3.smithi137.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 5/8 2024-05-21T12:32:00.579 INFO:tasks.workunit.client.3.smithi137.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 6/8 2024-05-21T12:32:00.737 INFO:tasks.workunit.client.3.smithi137.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 7/8 2024-05-21T12:32:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:00 smithi092 ceph-mon[33665]: pgmap v483: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:00.814 INFO:tasks.workunit.client.3.smithi137.stdout: Installing : fio-3.35-1.el9.x86_64 8/8 2024-05-21T12:32:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:00 smithi045 ceph-mon[30165]: pgmap v483: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:00.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:00 smithi072 ceph-mon[33912]: pgmap v483: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:02 smithi092 ceph-mon[33665]: pgmap v484: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:02.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:02 smithi045 ceph-mon[30165]: pgmap v484: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:02.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:02 smithi072 ceph-mon[33912]: pgmap v484: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:03.432 INFO:tasks.workunit.client.3.smithi137.stdout: Running scriptlet: fio-3.35-1.el9.x86_64 8/8 2024-05-21T12:32:03.432 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : fio-3.35-1.el9.x86_64 1/8 2024-05-21T12:32:03.432 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/8 2024-05-21T12:32:03.432 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/8 2024-05-21T12:32:03.432 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/8 2024-05-21T12:32:03.432 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/8 2024-05-21T12:32:03.432 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/8 2024-05-21T12:32:03.432 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/8 2024-05-21T12:32:03.765 INFO:tasks.workunit.client.3.smithi137.stdout: Verifying : libnbd-1.20.0-1.el9.x86_64 8/8 2024-05-21T12:32:03.766 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:03.766 INFO:tasks.workunit.client.3.smithi137.stdout:Installed: 2024-05-21T12:32:03.766 INFO:tasks.workunit.client.3.smithi137.stdout: fio-3.35-1.el9.x86_64 fio-engine-http-3.35-1.el9.x86_64 2024-05-21T12:32:03.766 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-libaio-3.35-1.el9.x86_64 fio-engine-nbd-3.35-1.el9.x86_64 2024-05-21T12:32:03.766 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-rados-3.35-1.el9.x86_64 fio-engine-rbd-3.35-1.el9.x86_64 2024-05-21T12:32:03.766 INFO:tasks.workunit.client.3.smithi137.stdout: fio-engine-rdma-3.35-1.el9.x86_64 libnbd-1.20.0-1.el9.x86_64 2024-05-21T12:32:03.766 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:03.766 INFO:tasks.workunit.client.3.smithi137.stdout:Complete! 2024-05-21T12:32:04.094 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo yum -y install sysstat 2024-05-21T12:32:04.620 INFO:tasks.workunit.client.3.smithi137.stdout:Last metadata expiration check: 0:19:29 ago on Tue 21 May 2024 12:12:35 PM UTC. 2024-05-21T12:32:04.699 INFO:tasks.workunit.client.3.smithi137.stdout:Package sysstat-12.5.4-7.el9.x86_64 is already installed. 2024-05-21T12:32:04.741 INFO:tasks.workunit.client.3.smithi137.stdout:Dependencies resolved. 2024-05-21T12:32:04.742 INFO:tasks.workunit.client.3.smithi137.stdout:Nothing to do. 2024-05-21T12:32:04.742 INFO:tasks.workunit.client.3.smithi137.stdout:Complete! 2024-05-21T12:32:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:04 smithi092 ceph-mon[33665]: pgmap v485: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:32:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:04 smithi045 ceph-mon[30165]: pgmap v485: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:32:04.817 INFO:tasks.workunit.client.3.smithi137.stderr:+ namespace_range_start= 2024-05-21T12:32:04.818 INFO:tasks.workunit.client.3.smithi137.stderr:+ namespace_range_end= 2024-05-21T12:32:04.818 INFO:tasks.workunit.client.3.smithi137.stderr:+ rbd_iostat=false 2024-05-21T12:32:04.818 INFO:tasks.workunit.client.3.smithi137.stderr:+ [[ 1 -gt 0 ]] 2024-05-21T12:32:04.818 INFO:tasks.workunit.client.3.smithi137.stderr:+ case "$1" in 2024-05-21T12:32:04.818 INFO:tasks.workunit.client.3.smithi137.stderr:+ rbd_iostat=true 2024-05-21T12:32:04.818 INFO:tasks.workunit.client.3.smithi137.stderr:+ shift 2024-05-21T12:32:04.818 INFO:tasks.workunit.client.3.smithi137.stderr:+ [[ 0 -gt 0 ]] 2024-05-21T12:32:04.818 INFO:tasks.workunit.client.3.smithi137.stderr:++ mktemp -t nvmeof-fio-XXXX 2024-05-21T12:32:04.819 INFO:tasks.workunit.client.3.smithi137.stderr:+ fio_file=/tmp/nvmeof-fio-vhMS 2024-05-21T12:32:04.819 INFO:tasks.workunit.client.3.smithi137.stderr:++ sudo nvme list --output-format=json 2024-05-21T12:32:04.819 INFO:tasks.workunit.client.3.smithi137.stderr:++ jq -r '.Devices | sort_by(.NameSpace) | .[] | select(.ModelNumber == "Ceph bdev Controller") | .DevicePath' 2024-05-21T12:32:04.844 INFO:tasks.workunit.client.3.smithi137.stderr:+ all_drives_list='/dev/nvme5n1 2024-05-21T12:32:04.844 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n1 2024-05-21T12:32:04.844 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n1 2024-05-21T12:32:04.844 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n2 2024-05-21T12:32:04.844 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n2 2024-05-21T12:32:04.844 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n2 2024-05-21T12:32:04.844 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n3 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n3 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n3 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n4 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n4 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n4 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n5 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n5 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n5 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n6 2024-05-21T12:32:04.845 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n6 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n6 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n7 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n7 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n7 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n8 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n8 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n8 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n9 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n9 2024-05-21T12:32:04.846 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n9 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n10 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n10 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n10 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n11 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n11 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n11 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n12 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n12 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n12 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n13 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n13 2024-05-21T12:32:04.847 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n13 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n14 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n14 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n14 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n15 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n15 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n15 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n16 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n16 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n16 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n17 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n17 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n17 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n18 2024-05-21T12:32:04.848 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n18 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n18 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n19 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n19 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n19 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n20 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n20 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n20' 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' '' ']' 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' '' ']' 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:+ selected_drives='/dev/nvme5n1 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n1 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n1 2024-05-21T12:32:04.849 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n2 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n2 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n2 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n3 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n3 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n3 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n4 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n4 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n4 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n5 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n5 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n5 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n6 2024-05-21T12:32:04.850 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n6 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n6 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n7 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n7 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n7 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n8 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n8 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n8 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n9 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n9 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n9 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n10 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n10 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n10 2024-05-21T12:32:04.851 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n11 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n11 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n11 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n12 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n12 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n12 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n13 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n13 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n13 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n14 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n14 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n14 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n15 2024-05-21T12:32:04.852 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n15 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n15 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n16 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n16 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n16 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n17 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n17 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n17 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n18 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n18 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n18 2024-05-21T12:32:04.853 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n19 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n19 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n19 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n20 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n20 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n20' 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:+ RUNTIME=600 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:+ cat 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:++ echo '/dev/nvme5n1 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n1 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n1 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n2 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n2 2024-05-21T12:32:04.854 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n2 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n3 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n3 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n3 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n4 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n4 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n4 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n5 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n5 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n5 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n6 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n6 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n6 2024-05-21T12:32:04.855 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n7 2024-05-21T12:32:04.856 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n7 2024-05-21T12:32:04.856 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n7 2024-05-21T12:32:04.856 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n8 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n8 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n8 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n9 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n9 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n9 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n10 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n10 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n10 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n11 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n11 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n11 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n12 2024-05-21T12:32:04.857 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n12 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n12 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n13 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n13 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n13 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n14 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n14 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n14 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n15 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n15 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n15 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n16 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n16 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n16 2024-05-21T12:32:04.858 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n17 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n17 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n17 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n18 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n18 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n18 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n19 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n19 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n19 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme5n20 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme2n20 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:/dev/nvme8n20' 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:++ tr '\n' : 2024-05-21T12:32:04.859 INFO:tasks.workunit.client.3.smithi137.stderr:++ sed 's/:$//' 2024-05-21T12:32:04.860 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof.fio] starting fio test...' 2024-05-21T12:32:04.860 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof.fio] starting fio test... 2024-05-21T12:32:04.861 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' -n 10 ']' 2024-05-21T12:32:04.862 INFO:tasks.workunit.client.3.smithi137.stderr:+ iostat_count=60 2024-05-21T12:32:04.862 INFO:tasks.workunit.client.3.smithi137.stderr:+ '[' true = true ']' 2024-05-21T12:32:04.862 INFO:tasks.workunit.client.3.smithi137.stderr:+ iterations=120 2024-05-21T12:32:04.862 INFO:tasks.workunit.client.3.smithi137.stderr:+ iostat -d -p /dev/nvme5n1 /dev/nvme2n1 /dev/nvme8n1 /dev/nvme5n2 /dev/nvme2n2 /dev/nvme8n2 /dev/nvme5n3 /dev/nvme2n3 /dev/nvme8n3 /dev/nvme5n4 /dev/nvme2n4 /dev/nvme8n4 /dev/nvme5n5 /dev/nvme2n5 /dev/nvme8n5 /dev/nvme5n6 /dev/nvme2n6 /dev/nvme8n6 /dev/nvme5n7 /dev/nvme2n7 /dev/nvme8n7 /dev/nvme5n8 /dev/nvme2n8 /dev/nvme8n8 /dev/nvme5n9 /dev/nvme2n9 /dev/nvme8n9 /dev/nvme5n10 /dev/nvme2n10 /dev/nvme8n10 /dev/nvme5n11 /dev/nvme2n11 /dev/nvme8n11 /dev/nvme5n12 /dev/nvme2n12 /dev/nvme8n12 /dev/nvme5n13 /dev/nvme2n13 /dev/nvme8n13 /dev/nvme5n14 /dev/nvme2n14 /dev/nvme8n14 /dev/nvme5n15 /dev/nvme2n15 /dev/nvme8n15 /dev/nvme5n16 /dev/nvme2n16 /dev/nvme8n16 /dev/nvme5n17 /dev/nvme2n17 /dev/nvme8n17 /dev/nvme5n18 /dev/nvme2n18 /dev/nvme8n18 /dev/nvme5n19 /dev/nvme2n19 /dev/nvme8n19 /dev/nvme5n20 /dev/nvme2n20 /dev/nvme8n20 10 60 -h 2024-05-21T12:32:04.862 INFO:tasks.workunit.client.3.smithi137.stderr:+ fio --showcmd /tmp/nvmeof-fio-vhMS 2024-05-21T12:32:04.862 INFO:tasks.workunit.client.3.smithi137.stderr:+ timeout 20 rbd perf image iostat mypool --iterations 120 2024-05-21T12:32:04.865 INFO:tasks.workunit.client.3.smithi137.stdout:Linux 5.14.0-447.el9.x86_64 (smithi137) 05/21/2024 _x86_64_ (8 CPU) 2024-05-21T12:32:04.865 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:32:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n1 2024-05-21T12:32:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n10 2024-05-21T12:32:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n11 2024-05-21T12:32:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n12 2024-05-21T12:32:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n13 2024-05-21T12:32:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n14 2024-05-21T12:32:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n15 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n16 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n17 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n18 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n19 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n2 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n20 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n3 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n4 2024-05-21T12:32:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n5 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n6 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n7 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n8 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme2n9 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n1 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n10 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n11 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n12 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n13 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n14 2024-05-21T12:32:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n15 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n16 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n17 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n18 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n19 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n2 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n20 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n3 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n4 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n5 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n6 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n7 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n8 2024-05-21T12:32:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme5n9 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n1 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n10 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n11 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n12 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n13 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n14 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n15 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n16 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n17 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n18 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n19 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n2 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n20 2024-05-21T12:32:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n3 2024-05-21T12:32:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n4 2024-05-21T12:32:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n5 2024-05-21T12:32:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n6 2024-05-21T12:32:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n7 2024-05-21T12:32:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n8 2024-05-21T12:32:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.03 0.7k 0.0k 0.0k 1.0M 0.0k 0.0k nvme8n9 2024-05-21T12:32:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:04.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:04 smithi072 ceph-mon[33912]: pgmap v485: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:32:04.988 INFO:tasks.workunit.client.3.smithi137.stderr:rbd: waiting for initial image stats 2024-05-21T12:32:04.988 INFO:tasks.workunit.client.3.smithi137.stderr: 2024-05-21T12:32:04.995 INFO:tasks.workunit.client.3.smithi137.stdout:fio --name=nvmeof-fio-test --ioengine=sync --bsrange=4k-64k --numjobs=1 --size=1G --time_based=1 --runtime=600 --rw=randrw --verify=md5 --verify_fatal=1 --direct=1 --filename=/dev/nvme5n1:/dev/nvme2n1:/dev/nvme8n1:/dev/nvme5n2:/dev/nvme2n2:/dev/nvme8n2:/dev/nvme5n3:/dev/nvme2n3:/dev/nvme8n3:/dev/nvme5n4:/dev/nvme2n4:/dev/nvme8n4:/dev/nvme5n5:/dev/nvme2n5:/dev/nvme8n5:/dev/nvme5n6:/dev/nvme2n6:/dev/nvme8n6:/dev/nvme5n7:/dev/nvme2n7:/dev/nvme8n7:/dev/nvme5n8:/dev/nvme2n8:/dev/nvme8n8:/dev/nvme5n9:/dev/nvme2n9:/dev/nvme8n9:/dev/nvme5n10:/dev/nvme2n10:/dev/nvme8n10:/dev/nvme5n11:/dev/nvme2n11:/dev/nvme8n11:/dev/nvme5n12:/dev/nvme2n12:/dev/nvme8n12:/dev/nvme5n13:/dev/nvme2n13:/dev/nvme8n13:/dev/nvme5n14:/dev/nvme2n14:/dev/nvme8n14:/dev/nvme5n15:/dev/nvme2n15:/dev/nvme8n15:/dev/nvme5n16:/dev/nvme2n16:/dev/nvme8n16:/dev/nvme5n17:/dev/nvme2n17:/dev/nvme8n17:/dev/nvme5n18:/dev/nvme2n18:/dev/nvme8n18:/dev/nvme5n19:/dev/nvme2n19:/dev/nvme8n19:/dev/nvme5n20:/dev/nvme2n20:/dev/nvme8n20 2024-05-21T12:32:04.995 INFO:tasks.workunit.client.3.smithi137.stderr:+ sudo fio /tmp/nvmeof-fio-vhMS 2024-05-21T12:32:05.142 INFO:tasks.workunit.client.3.smithi137.stdout:nvmeof-fio-test: (g=0): rw=randrw, bs=(R) 4096B-64.0KiB, (W) 4096B-64.0KiB, (T) 4096B-64.0KiB, ioengine=sync, iodepth=1 2024-05-21T12:32:05.142 INFO:tasks.workunit.client.3.smithi137.stdout:fio-3.35 2024-05-21T12:32:05.142 INFO:tasks.workunit.client.3.smithi137.stdout:Starting 1 process 2024-05-21T12:32:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:06 smithi092 ceph-mon[33665]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:06 smithi092 ceph-mon[33665]: { 2024-05-21T12:32:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:06 smithi092 ceph-mon[33665]: "prefix": "rbd perf image stats", 2024-05-21T12:32:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:06 smithi092 ceph-mon[33665]: "pool_spec": "mypool", 2024-05-21T12:32:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:06 smithi092 ceph-mon[33665]: "sort_by": "write_ops", 2024-05-21T12:32:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:06 smithi092 ceph-mon[33665]: "format": "json" 2024-05-21T12:32:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:06 smithi092 ceph-mon[33665]: }"]: dispatch 2024-05-21T12:32:06.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:06 smithi092 ceph-mon[33665]: pgmap v486: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:06 smithi045 ceph-mon[30165]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:06 smithi045 ceph-mon[30165]: { 2024-05-21T12:32:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:06 smithi045 ceph-mon[30165]: "prefix": "rbd perf image stats", 2024-05-21T12:32:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:06 smithi045 ceph-mon[30165]: "pool_spec": "mypool", 2024-05-21T12:32:06.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:06 smithi045 ceph-mon[30165]: "sort_by": "write_ops", 2024-05-21T12:32:06.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:06 smithi045 ceph-mon[30165]: "format": "json" 2024-05-21T12:32:06.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:06 smithi045 ceph-mon[30165]: }"]: dispatch 2024-05-21T12:32:06.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:06 smithi045 ceph-mon[30165]: pgmap v486: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:06.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:06 smithi072 ceph-mon[33912]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:06.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:06 smithi072 ceph-mon[33912]: { 2024-05-21T12:32:06.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:06 smithi072 ceph-mon[33912]: "prefix": "rbd perf image stats", 2024-05-21T12:32:06.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:06 smithi072 ceph-mon[33912]: "pool_spec": "mypool", 2024-05-21T12:32:06.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:06 smithi072 ceph-mon[33912]: "sort_by": "write_ops", 2024-05-21T12:32:06.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:06 smithi072 ceph-mon[33912]: "format": "json" 2024-05-21T12:32:06.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:06 smithi072 ceph-mon[33912]: }"]: dispatch 2024-05-21T12:32:06.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:06 smithi072 ceph-mon[33912]: pgmap v486: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:32:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:08 smithi092 ceph-mon[33665]: pgmap v487: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-05-21T12:32:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:08 smithi045 ceph-mon[30165]: pgmap v487: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-05-21T12:32:08.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:08 smithi072 ceph-mon[33912]: pgmap v487: 33 pgs: 33 active+clean; 609 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-05-21T12:32:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:10 smithi092 ceph-mon[33665]: pgmap v488: 33 pgs: 33 active+clean; 19 MiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-05-21T12:32:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:10 smithi045 ceph-mon[30165]: pgmap v488: 33 pgs: 33 active+clean; 19 MiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-05-21T12:32:10.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:10 smithi072 ceph-mon[33912]: pgmap v488: 33 pgs: 33 active+clean; 19 MiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-05-21T12:32:11.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:11 smithi092 ceph-mon[33665]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:11.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:11 smithi092 ceph-mon[33665]: { 2024-05-21T12:32:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:11 smithi092 ceph-mon[33665]: "prefix": "rbd perf image stats", 2024-05-21T12:32:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:11 smithi092 ceph-mon[33665]: "pool_spec": "mypool", 2024-05-21T12:32:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:11 smithi092 ceph-mon[33665]: "sort_by": "write_ops", 2024-05-21T12:32:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:11 smithi092 ceph-mon[33665]: "format": "json" 2024-05-21T12:32:11.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:11 smithi092 ceph-mon[33665]: }"]: dispatch 2024-05-21T12:32:11.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:11 smithi045 ceph-mon[30165]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:11.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:11 smithi045 ceph-mon[30165]: { 2024-05-21T12:32:11.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:11 smithi045 ceph-mon[30165]: "prefix": "rbd perf image stats", 2024-05-21T12:32:11.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:11 smithi045 ceph-mon[30165]: "pool_spec": "mypool", 2024-05-21T12:32:11.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:11 smithi045 ceph-mon[30165]: "sort_by": "write_ops", 2024-05-21T12:32:11.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:11 smithi045 ceph-mon[30165]: "format": "json" 2024-05-21T12:32:11.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:11 smithi045 ceph-mon[30165]: }"]: dispatch 2024-05-21T12:32:11.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:11 smithi072 ceph-mon[33912]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:11 smithi072 ceph-mon[33912]: { 2024-05-21T12:32:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:11 smithi072 ceph-mon[33912]: "prefix": "rbd perf image stats", 2024-05-21T12:32:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:11 smithi072 ceph-mon[33912]: "pool_spec": "mypool", 2024-05-21T12:32:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:11 smithi072 ceph-mon[33912]: "sort_by": "write_ops", 2024-05-21T12:32:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:11 smithi072 ceph-mon[33912]: "format": "json" 2024-05-21T12:32:11.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:11 smithi072 ceph-mon[33912]: }"]: dispatch 2024-05-21T12:32:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:12 smithi092 ceph-mon[33665]: pgmap v489: 33 pgs: 33 active+clean; 121 MiB data, 238 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 261 KiB/s wr, 60 op/s 2024-05-21T12:32:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:12 smithi045 ceph-mon[30165]: pgmap v489: 33 pgs: 33 active+clean; 121 MiB data, 238 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 261 KiB/s wr, 60 op/s 2024-05-21T12:32:12.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:12 smithi072 ceph-mon[33912]: pgmap v489: 33 pgs: 33 active+clean; 121 MiB data, 238 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s rd, 261 KiB/s wr, 60 op/s 2024-05-21T12:32:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:14 smithi092 ceph-mon[33665]: pgmap v490: 33 pgs: 33 active+clean; 609 MiB data, 307 MiB used, 715 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 331 op/s 2024-05-21T12:32:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:14 smithi045 ceph-mon[30165]: pgmap v490: 33 pgs: 33 active+clean; 609 MiB data, 307 MiB used, 715 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 331 op/s 2024-05-21T12:32:14.865 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:14.872 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:32:14.872 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 249.6k 135.6k 0.0k 2.4M 1.3M 0.0k nvme2n1 2024-05-21T12:32:14.872 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 266.0k 112.4k 0.0k 2.6M 1.1M 0.0k nvme2n10 2024-05-21T12:32:14.872 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 233.6k 126.0k 0.0k 2.3M 1.2M 0.0k nvme2n11 2024-05-21T12:32:14.872 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 250.8k 121.6k 0.0k 2.4M 1.2M 0.0k nvme2n12 2024-05-21T12:32:14.872 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 261.6k 108.8k 0.0k 2.6M 1.1M 0.0k nvme2n13 2024-05-21T12:32:14.872 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 233.6k 134.8k 0.0k 2.3M 1.3M 0.0k nvme2n14 2024-05-21T12:32:14.872 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 254.0k 97.6k 0.0k 2.5M 976.0k 0.0k nvme2n15 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 246.0k 112.8k 0.0k 2.4M 1.1M 0.0k nvme2n16 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 197.6k 139.2k 0.0k 1.9M 1.4M 0.0k nvme2n17 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 221.6k 144.0k 0.0k 2.2M 1.4M 0.0k nvme2n18 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 198.0k 176.0k 0.0k 1.9M 1.7M 0.0k nvme2n19 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 237.2k 133.6k 0.0k 2.3M 1.3M 0.0k nvme2n2 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 249.6k 112.0k 0.0k 2.4M 1.1M 0.0k nvme2n20 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 226.0k 130.0k 0.0k 2.2M 1.3M 0.0k nvme2n3 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 250.0k 128.0k 0.0k 2.4M 1.2M 0.0k nvme2n4 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 241.2k 106.0k 0.0k 2.4M 1.0M 0.0k nvme2n5 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 210.4k 166.8k 0.0k 2.1M 1.6M 0.0k nvme2n6 2024-05-21T12:32:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 250.8k 129.2k 0.0k 2.4M 1.3M 0.0k nvme2n7 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 241.6k 111.2k 0.0k 2.4M 1.1M 0.0k nvme2n8 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 212.0k 172.8k 0.0k 2.1M 1.7M 0.0k nvme2n9 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 203.2k 150.0k 0.0k 2.0M 1.5M 0.0k nvme5n1 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 219.6k 146.8k 0.0k 2.1M 1.4M 0.0k nvme5n10 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 218.8k 128.0k 0.0k 2.1M 1.2M 0.0k nvme5n11 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 223.6k 125.2k 0.0k 2.2M 1.2M 0.0k nvme5n12 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 215.2k 159.6k 0.0k 2.1M 1.6M 0.0k nvme5n13 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 207.6k 132.0k 0.0k 2.0M 1.3M 0.0k nvme5n14 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 273.2k 117.6k 0.0k 2.7M 1.1M 0.0k nvme5n15 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 276.0k 124.4k 0.0k 2.7M 1.2M 0.0k nvme5n16 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 246.8k 126.0k 0.0k 2.4M 1.2M 0.0k nvme5n17 2024-05-21T12:32:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 226.0k 140.4k 0.0k 2.2M 1.4M 0.0k nvme5n18 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 246.8k 134.0k 0.0k 2.4M 1.3M 0.0k nvme5n19 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 205.6k 162.0k 0.0k 2.0M 1.6M 0.0k nvme5n2 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 235.2k 128.0k 0.0k 2.3M 1.2M 0.0k nvme5n20 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 234.0k 104.0k 0.0k 2.3M 1.0M 0.0k nvme5n3 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 259.6k 112.0k 0.0k 2.5M 1.1M 0.0k nvme5n4 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 231.2k 133.2k 0.0k 2.3M 1.3M 0.0k nvme5n5 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 250.8k 136.4k 0.0k 2.4M 1.3M 0.0k nvme5n6 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 238.4k 131.2k 0.0k 2.3M 1.3M 0.0k nvme5n7 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 219.2k 152.0k 0.0k 2.1M 1.5M 0.0k nvme5n8 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 244.8k 147.2k 0.0k 2.4M 1.4M 0.0k nvme5n9 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 230.4k 110.8k 0.0k 2.2M 1.1M 0.0k nvme8n1 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 214.8k 139.6k 0.0k 2.1M 1.4M 0.0k nvme8n10 2024-05-21T12:32:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 254.0k 142.0k 0.0k 2.5M 1.4M 0.0k nvme8n11 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 236.0k 137.6k 0.0k 2.3M 1.3M 0.0k nvme8n12 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 222.8k 114.8k 0.0k 2.2M 1.1M 0.0k nvme8n13 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 257.2k 162.0k 0.0k 2.5M 1.6M 0.0k nvme8n14 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 220.8k 126.8k 0.0k 2.2M 1.2M 0.0k nvme8n15 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 242.0k 151.2k 0.0k 2.4M 1.5M 0.0k nvme8n16 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 248.8k 144.4k 0.0k 2.4M 1.4M 0.0k nvme8n17 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 227.2k 158.4k 0.0k 2.2M 1.5M 0.0k nvme8n18 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 241.2k 115.6k 0.0k 2.4M 1.1M 0.0k nvme8n19 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 226.4k 138.8k 0.0k 2.2M 1.4M 0.0k nvme8n2 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.20 216.4k 146.8k 0.0k 2.1M 1.4M 0.0k nvme8n20 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 244.4k 160.0k 0.0k 2.4M 1.6M 0.0k nvme8n3 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 192.4k 152.8k 0.0k 1.9M 1.5M 0.0k nvme8n4 2024-05-21T12:32:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 223.6k 162.0k 0.0k 2.2M 1.6M 0.0k nvme8n5 2024-05-21T12:32:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 251.6k 110.0k 0.0k 2.5M 1.1M 0.0k nvme8n6 2024-05-21T12:32:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 233.2k 138.4k 0.0k 2.3M 1.4M 0.0k nvme8n7 2024-05-21T12:32:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 244.0k 135.6k 0.0k 2.4M 1.3M 0.0k nvme8n8 2024-05-21T12:32:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.30 225.6k 130.8k 0.0k 2.2M 1.3M 0.0k nvme8n9 2024-05-21T12:32:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:14.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:14 smithi072 ceph-mon[33912]: pgmap v490: 33 pgs: 33 active+clean; 609 MiB data, 307 MiB used, 715 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 331 op/s 2024-05-21T12:32:14.993 INFO:tasks.workunit.client.3.smithi137.stdout:NAME WR RD WR_BYTES RD_BYTES WR_LAT RD_LAT 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage6 8/s 4/s 251 KiB/s 112 KiB/s 943.42 us 158.36 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage22 7/s 4/s 264 KiB/s 48 KiB/s 970.93 us 151.00 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage43 7/s 6/s 284 KiB/s 126 KiB/s 956.43 us 153.37 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage51 6/s 4/s 201 KiB/s 135 KiB/s 936.48 us 156.01 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage58 6/s 4/s 218 KiB/s 72 KiB/s 969.29 us 166.41 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage23 6/s 7/s 174 KiB/s 110 KiB/s 913.32 us 163.88 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage19 6/s 2/s 233 KiB/s 51 KiB/s 940.85 us 169.94 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage33 6/s 4/s 215 KiB/s 90 KiB/s 932.12 us 171.09 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage37 6/s 5/s 180 KiB/s 99 KiB/s 952.51 us 163.58 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage53 6/s 4/s 148 KiB/s 124 KiB/s 917.43 us 146.41 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage12 5/s 4/s 164 KiB/s 73 KiB/s 934.82 us 151.13 us 2024-05-21T12:32:14.994 INFO:tasks.workunit.client.3.smithi137.stdout:myimage44 5/s 4/s 191 KiB/s 61 KiB/s 973.49 us 165.67 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage1 5/s 5/s 194 KiB/s 134 KiB/s 925.72 us 170.77 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage2 5/s 6/s 180 KiB/s 174 KiB/s 948.00 us 157.85 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage3 5/s 4/s 156 KiB/s 95 KiB/s 932.10 us 145.45 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage21 5/s 3/s 170 KiB/s 54 KiB/s 944.27 us 159.91 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage36 5/s 6/s 176 KiB/s 180 KiB/s 944.76 us 165.29 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage39 5/s 3/s 185 KiB/s 107 KiB/s 1.07 ms 163.88 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage46 5/s 5/s 166 KiB/s 144 KiB/s 970.60 us 161.05 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage47 5/s 5/s 195 KiB/s 131 KiB/s 968.37 us 175.37 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage48 5/s 3/s 182 KiB/s 50 KiB/s 932.76 us 151.68 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage52 5/s 4/s 132 KiB/s 78 KiB/s 959.01 us 152.15 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage7 5/s 4/s 162 KiB/s 82 KiB/s 962.58 us 154.26 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage10 5/s 4/s 180 KiB/s 82 KiB/s 947.87 us 152.05 us 2024-05-21T12:32:14.995 INFO:tasks.workunit.client.3.smithi137.stdout:myimage26 5/s 4/s 170 KiB/s 151 KiB/s 939.35 us 164.18 us 2024-05-21T12:32:14.996 INFO:tasks.workunit.client.3.smithi137.stdout:myimage41 5/s 3/s 129 KiB/s 94 KiB/s 914.35 us 150.32 us 2024-05-21T12:32:14.996 INFO:tasks.workunit.client.3.smithi137.stdout:myimage25 4/s 3/s 164 KiB/s 52 KiB/s 962.70 us 169.28 us 2024-05-21T12:32:14.996 INFO:tasks.workunit.client.3.smithi137.stdout:myimage57 4/s 2/s 150 KiB/s 58 KiB/s 957.65 us 146.13 us 2024-05-21T12:32:14.996 INFO:tasks.workunit.client.3.smithi137.stdout:myimage35 4/s 5/s 164 KiB/s 102 KiB/s 953.61 us 150.54 us 2024-05-21T12:32:14.996 INFO:tasks.workunit.client.3.smithi137.stdout:myimage14 4/s 4/s 130 KiB/s 86 KiB/s 910.42 us 152.71 us 2024-05-21T12:32:14.996 INFO:tasks.workunit.client.3.smithi137.stdout:myimage42 4/s 5/s 143 KiB/s 102 KiB/s 956.81 us 166.30 us 2024-05-21T12:32:14.996 INFO:tasks.workunit.client.3.smithi137.stdout:myimage20 3/s 3/s 104 KiB/s 42 KiB/s 907.17 us 182.26 us 2024-05-21T12:32:14.996 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:16 smithi092 ceph-mon[33665]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:16 smithi092 ceph-mon[33665]: { 2024-05-21T12:32:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:16 smithi092 ceph-mon[33665]: "prefix": "rbd perf image stats", 2024-05-21T12:32:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:16 smithi092 ceph-mon[33665]: "pool_spec": "mypool", 2024-05-21T12:32:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:16 smithi092 ceph-mon[33665]: "sort_by": "write_ops", 2024-05-21T12:32:16.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:16 smithi092 ceph-mon[33665]: "format": "json" 2024-05-21T12:32:16.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:16 smithi092 ceph-mon[33665]: }"]: dispatch 2024-05-21T12:32:16.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:16 smithi092 ceph-mon[33665]: pgmap v491: 33 pgs: 33 active+clean; 845 MiB data, 382 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 492 op/s 2024-05-21T12:32:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:16 smithi045 ceph-mon[30165]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:16 smithi045 ceph-mon[30165]: { 2024-05-21T12:32:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:16 smithi045 ceph-mon[30165]: "prefix": "rbd perf image stats", 2024-05-21T12:32:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:16 smithi045 ceph-mon[30165]: "pool_spec": "mypool", 2024-05-21T12:32:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:16 smithi045 ceph-mon[30165]: "sort_by": "write_ops", 2024-05-21T12:32:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:16 smithi045 ceph-mon[30165]: "format": "json" 2024-05-21T12:32:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:16 smithi045 ceph-mon[30165]: }"]: dispatch 2024-05-21T12:32:16.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:16 smithi045 ceph-mon[30165]: pgmap v491: 33 pgs: 33 active+clean; 845 MiB data, 382 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 492 op/s 2024-05-21T12:32:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:16 smithi072 ceph-mon[33912]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:16 smithi072 ceph-mon[33912]: { 2024-05-21T12:32:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:16 smithi072 ceph-mon[33912]: "prefix": "rbd perf image stats", 2024-05-21T12:32:16.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:16 smithi072 ceph-mon[33912]: "pool_spec": "mypool", 2024-05-21T12:32:16.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:16 smithi072 ceph-mon[33912]: "sort_by": "write_ops", 2024-05-21T12:32:16.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:16 smithi072 ceph-mon[33912]: "format": "json" 2024-05-21T12:32:16.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:16 smithi072 ceph-mon[33912]: }"]: dispatch 2024-05-21T12:32:16.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:16 smithi072 ceph-mon[33912]: pgmap v491: 33 pgs: 33 active+clean; 845 MiB data, 382 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 492 op/s 2024-05-21T12:32:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:18 smithi092 ceph-mon[33665]: pgmap v492: 33 pgs: 33 active+clean; 915 MiB data, 474 MiB used, 715 GiB / 715 GiB avail; 4.2 MiB/s rd, 6.9 MiB/s wr, 665 op/s 2024-05-21T12:32:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:18 smithi045 ceph-mon[30165]: pgmap v492: 33 pgs: 33 active+clean; 915 MiB data, 474 MiB used, 715 GiB / 715 GiB avail; 4.2 MiB/s rd, 6.9 MiB/s wr, 665 op/s 2024-05-21T12:32:18.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:18 smithi072 ceph-mon[33912]: pgmap v492: 33 pgs: 33 active+clean; 915 MiB data, 474 MiB used, 715 GiB / 715 GiB avail; 4.2 MiB/s rd, 6.9 MiB/s wr, 665 op/s 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:NAME WR RD WR_BYTES RD_BYTES WR_LAT RD_LAT 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:myimage58 9/s 6/s 291 KiB/s 154 KiB/s 989.49 us 168.35 us 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:myimage39 9/s 7/s 282 KiB/s 176 KiB/s 997.73 us 163.81 us 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:myimage25 8/s 7/s 213 KiB/s 194 KiB/s 948.24 us 161.83 us 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:myimage28 8/s 6/s 286 KiB/s 125 KiB/s 958.02 us 164.60 us 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:myimage12 8/s 6/s 282 KiB/s 151 KiB/s 1.13 ms 156.53 us 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:myimage19 8/s 7/s 284 KiB/s 212 KiB/s 955.75 us 167.08 us 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:myimage15 8/s 8/s 261 KiB/s 212 KiB/s 954.80 us 154.48 us 2024-05-21T12:32:19.998 INFO:tasks.workunit.client.3.smithi137.stdout:myimage30 8/s 6/s 225 KiB/s 142 KiB/s 969.60 us 159.36 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage34 8/s 6/s 247 KiB/s 182 KiB/s 927.20 us 156.09 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage1 8/s 7/s 243 KiB/s 198 KiB/s 956.52 us 163.12 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage2 8/s 8/s 234 KiB/s 160 KiB/s 958.07 us 160.19 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage41 8/s 6/s 216 KiB/s 154 KiB/s 962.90 us 194.00 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage44 8/s 6/s 249 KiB/s 148 KiB/s 956.19 us 182.46 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage4 8/s 6/s 235 KiB/s 192 KiB/s 948.48 us 160.56 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage55 8/s 7/s 251 KiB/s 148 KiB/s 978.74 us 163.38 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage6 8/s 7/s 203 KiB/s 244 KiB/s 974.79 us 151.38 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage11 8/s 7/s 253 KiB/s 186 KiB/s 968.60 us 192.25 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage56 8/s 7/s 290 KiB/s 153 KiB/s 1.02 ms 186.97 us 2024-05-21T12:32:19.999 INFO:tasks.workunit.client.3.smithi137.stdout:myimage59 8/s 7/s 281 KiB/s 214 KiB/s 984.13 us 158.40 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage22 8/s 8/s 233 KiB/s 239 KiB/s 962.43 us 167.96 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage48 8/s 7/s 212 KiB/s 174 KiB/s 944.91 us 176.57 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage8 8/s 6/s 233 KiB/s 135 KiB/s 948.22 us 178.96 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage18 8/s 7/s 220 KiB/s 175 KiB/s 979.08 us 155.27 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage31 8/s 7/s 238 KiB/s 180 KiB/s 976.87 us 151.45 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage40 8/s 7/s 246 KiB/s 132 KiB/s 972.56 us 171.31 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage45 8/s 7/s 272 KiB/s 177 KiB/s 977.36 us 181.42 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage21 7/s 7/s 244 KiB/s 158 KiB/s 967.19 us 157.71 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage52 7/s 7/s 255 KiB/s 193 KiB/s 974.89 us 166.09 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage35 7/s 5/s 232 KiB/s 113 KiB/s 993.85 us 164.00 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage5 7/s 7/s 254 KiB/s 194 KiB/s 945.23 us 166.44 us 2024-05-21T12:32:20.000 INFO:tasks.workunit.client.3.smithi137.stdout:myimage16 7/s 8/s 240 KiB/s 227 KiB/s 940.74 us 159.93 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage27 7/s 6/s 244 KiB/s 159 KiB/s 965.06 us 174.97 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage17 7/s 7/s 218 KiB/s 164 KiB/s 983.41 us 163.22 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage29 7/s 7/s 232 KiB/s 188 KiB/s 962.89 us 166.00 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage49 7/s 7/s 246 KiB/s 179 KiB/s 1.01 ms 176.75 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage26 7/s 7/s 214 KiB/s 167 KiB/s 933.56 us 168.34 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage38 7/s 8/s 219 KiB/s 191 KiB/s 941.83 us 173.90 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage9 7/s 7/s 290 KiB/s 200 KiB/s 943.29 us 157.46 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage10 7/s 9/s 219 KiB/s 282 KiB/s 1.01 ms 155.28 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage14 7/s 8/s 246 KiB/s 213 KiB/s 944.37 us 163.24 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage42 7/s 7/s 213 KiB/s 237 KiB/s 970.62 us 195.83 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage32 7/s 7/s 213 KiB/s 162 KiB/s 971.68 us 182.85 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage50 7/s 7/s 225 KiB/s 231 KiB/s 934.95 us 160.41 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage60 7/s 8/s 205 KiB/s 207 KiB/s 944.00 us 165.16 us 2024-05-21T12:32:20.001 INFO:tasks.workunit.client.3.smithi137.stdout:myimage7 7/s 7/s 206 KiB/s 169 KiB/s 990.72 us 159.62 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage53 7/s 7/s 229 KiB/s 203 KiB/s 955.56 us 155.20 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage57 7/s 6/s 220 KiB/s 164 KiB/s 978.71 us 174.70 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage24 7/s 7/s 158 KiB/s 184 KiB/s 979.21 us 166.83 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage33 6/s 9/s 244 KiB/s 234 KiB/s 949.54 us 177.18 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage47 6/s 8/s 210 KiB/s 178 KiB/s 985.61 us 161.28 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage13 6/s 9/s 162 KiB/s 226 KiB/s 980.77 us 162.41 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage54 6/s 7/s 230 KiB/s 189 KiB/s 987.19 us 176.71 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage3 6/s 7/s 190 KiB/s 174 KiB/s 971.73 us 170.78 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage36 6/s 7/s 223 KiB/s 218 KiB/s 981.32 us 160.39 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage46 6/s 7/s 180 KiB/s 223 KiB/s 981.92 us 173.66 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage43 6/s 9/s 137 KiB/s 227 KiB/s 991.04 us 161.75 us 2024-05-21T12:32:20.002 INFO:tasks.workunit.client.3.smithi137.stdout:myimage20 6/s 8/s 190 KiB/s 176 KiB/s 960.47 us 174.32 us 2024-05-21T12:32:20.003 INFO:tasks.workunit.client.3.smithi137.stdout:myimage51 6/s 8/s 218 KiB/s 188 KiB/s 1.00 ms 180.28 us 2024-05-21T12:32:20.003 INFO:tasks.workunit.client.3.smithi137.stdout:myimage23 5/s 10/s 158 KiB/s 270 KiB/s 958.21 us 161.18 us 2024-05-21T12:32:20.003 INFO:tasks.workunit.client.3.smithi137.stdout:myimage37 4/s 10/s 146 KiB/s 290 KiB/s 1.01 ms 157.51 us 2024-05-21T12:32:20.003 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:20.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:20 smithi092 ceph-mon[33665]: pgmap v493: 33 pgs: 33 active+clean; 948 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 6.1 MiB/s rd, 9.1 MiB/s wr, 811 op/s 2024-05-21T12:32:20.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:20 smithi045 ceph-mon[30165]: pgmap v493: 33 pgs: 33 active+clean; 948 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 6.1 MiB/s rd, 9.1 MiB/s wr, 811 op/s 2024-05-21T12:32:20.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:20 smithi072 ceph-mon[33912]: pgmap v493: 33 pgs: 33 active+clean; 948 MiB data, 566 MiB used, 715 GiB / 715 GiB avail; 6.1 MiB/s rd, 9.1 MiB/s wr, 811 op/s 2024-05-21T12:32:21.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:21 smithi092 ceph-mon[33665]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:21.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:21 smithi092 ceph-mon[33665]: { 2024-05-21T12:32:21.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:21 smithi092 ceph-mon[33665]: "prefix": "rbd perf image stats", 2024-05-21T12:32:21.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:21 smithi092 ceph-mon[33665]: "pool_spec": "mypool", 2024-05-21T12:32:21.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:21 smithi092 ceph-mon[33665]: "sort_by": "write_ops", 2024-05-21T12:32:21.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:21 smithi092 ceph-mon[33665]: "format": "json" 2024-05-21T12:32:21.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:21 smithi092 ceph-mon[33665]: }"]: dispatch 2024-05-21T12:32:21.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:21 smithi045 ceph-mon[30165]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:21.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:21 smithi045 ceph-mon[30165]: { 2024-05-21T12:32:21.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:21 smithi045 ceph-mon[30165]: "prefix": "rbd perf image stats", 2024-05-21T12:32:21.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:21 smithi045 ceph-mon[30165]: "pool_spec": "mypool", 2024-05-21T12:32:21.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:21 smithi045 ceph-mon[30165]: "sort_by": "write_ops", 2024-05-21T12:32:21.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:21 smithi045 ceph-mon[30165]: "format": "json" 2024-05-21T12:32:21.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:21 smithi045 ceph-mon[30165]: }"]: dispatch 2024-05-21T12:32:21.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:21 smithi072 ceph-mon[33912]: from='client.24946 -' entity='client.admin' cmd=[ 2024-05-21T12:32:21.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:21 smithi072 ceph-mon[33912]: { 2024-05-21T12:32:21.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:21 smithi072 ceph-mon[33912]: "prefix": "rbd perf image stats", 2024-05-21T12:32:21.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:21 smithi072 ceph-mon[33912]: "pool_spec": "mypool", 2024-05-21T12:32:21.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:21 smithi072 ceph-mon[33912]: "sort_by": "write_ops", 2024-05-21T12:32:21.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:21 smithi072 ceph-mon[33912]: "format": "json" 2024-05-21T12:32:21.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:21 smithi072 ceph-mon[33912]: }"]: dispatch 2024-05-21T12:32:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:22 smithi092 ceph-mon[33665]: pgmap v494: 33 pgs: 33 active+clean; 963 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 878 op/s 2024-05-21T12:32:22.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:22 smithi045 ceph-mon[30165]: pgmap v494: 33 pgs: 33 active+clean; 963 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 878 op/s 2024-05-21T12:32:22.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:22 smithi072 ceph-mon[33912]: pgmap v494: 33 pgs: 33 active+clean; 963 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 878 op/s 2024-05-21T12:32:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:24 smithi092 ceph-mon[33665]: pgmap v495: 33 pgs: 33 active+clean; 976 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 9.8 MiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-05-21T12:32:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:24 smithi045 ceph-mon[30165]: pgmap v495: 33 pgs: 33 active+clean; 976 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 9.8 MiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-05-21T12:32:24.865 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:32:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 204.8k 257.2k 0.0k 2.0M 2.5M 0.0k nvme2n1 2024-05-21T12:32:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 242.4k 230.4k 0.0k 2.4M 2.2M 0.0k nvme2n10 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 245.6k 256.4k 0.0k 2.4M 2.5M 0.0k nvme2n11 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 183.2k 259.6k 0.0k 1.8M 2.5M 0.0k nvme2n12 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 268.0k 199.6k 0.0k 2.6M 1.9M 0.0k nvme2n13 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 219.6k 241.2k 0.0k 2.1M 2.4M 0.0k nvme2n14 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 237.6k 223.6k 0.0k 2.3M 2.2M 0.0k nvme2n15 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 216.8k 208.8k 0.0k 2.1M 2.0M 0.0k nvme2n16 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 243.2k 234.4k 0.0k 2.4M 2.3M 0.0k nvme2n17 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 196.8k 247.6k 0.0k 1.9M 2.4M 0.0k nvme2n18 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 245.6k 215.2k 0.0k 2.4M 2.1M 0.0k nvme2n19 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 206.8k 225.6k 0.0k 2.0M 2.2M 0.0k nvme2n2 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 232.4k 256.4k 0.0k 2.3M 2.5M 0.0k nvme2n20 2024-05-21T12:32:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 203.2k 240.8k 0.0k 2.0M 2.4M 0.0k nvme2n3 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 230.8k 215.2k 0.0k 2.3M 2.1M 0.0k nvme2n4 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 277.6k 176.8k 0.0k 2.7M 1.7M 0.0k nvme2n5 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 236.4k 215.6k 0.0k 2.3M 2.1M 0.0k nvme2n6 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 238.8k 225.6k 0.0k 2.3M 2.2M 0.0k nvme2n7 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 241.6k 200.4k 0.0k 2.4M 2.0M 0.0k nvme2n8 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 244.8k 225.2k 0.0k 2.4M 2.2M 0.0k nvme2n9 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 224.4k 264.8k 0.0k 2.2M 2.6M 0.0k nvme5n1 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 244.8k 200.0k 0.0k 2.4M 2.0M 0.0k nvme5n10 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 222.4k 200.4k 0.0k 2.2M 2.0M 0.0k nvme5n11 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 230.4k 224.8k 0.0k 2.2M 2.2M 0.0k nvme5n12 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 242.4k 213.6k 0.0k 2.4M 2.1M 0.0k nvme5n13 2024-05-21T12:32:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 219.2k 246.0k 0.0k 2.1M 2.4M 0.0k nvme5n14 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 214.8k 210.8k 0.0k 2.1M 2.1M 0.0k nvme5n15 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 243.2k 203.6k 0.0k 2.4M 2.0M 0.0k nvme5n16 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 252.4k 220.4k 0.0k 2.5M 2.2M 0.0k nvme5n17 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 197.6k 227.6k 0.0k 1.9M 2.2M 0.0k nvme5n18 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 210.0k 256.0k 0.0k 2.1M 2.5M 0.0k nvme5n19 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 236.4k 208.8k 0.0k 2.3M 2.0M 0.0k nvme5n2 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 246.0k 197.2k 0.0k 2.4M 1.9M 0.0k nvme5n20 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 250.8k 202.4k 0.0k 2.4M 2.0M 0.0k nvme5n3 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 245.6k 203.6k 0.0k 2.4M 2.0M 0.0k nvme5n4 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 242.0k 219.6k 0.0k 2.4M 2.1M 0.0k nvme5n5 2024-05-21T12:32:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 199.6k 217.2k 0.0k 1.9M 2.1M 0.0k nvme5n6 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 220.4k 236.4k 0.0k 2.2M 2.3M 0.0k nvme5n7 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 148.0k 269.6k 0.0k 1.4M 2.6M 0.0k nvme5n8 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 206.8k 236.8k 0.0k 2.0M 2.3M 0.0k nvme5n9 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 221.2k 255.2k 0.0k 2.2M 2.5M 0.0k nvme8n1 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 245.6k 197.6k 0.0k 2.4M 1.9M 0.0k nvme8n10 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 213.6k 247.6k 0.0k 2.1M 2.4M 0.0k nvme8n11 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 193.6k 244.0k 0.0k 1.9M 2.4M 0.0k nvme8n12 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 219.2k 209.6k 0.0k 2.1M 2.0M 0.0k nvme8n13 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 213.2k 234.0k 0.0k 2.1M 2.3M 0.0k nvme8n14 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 224.0k 230.4k 0.0k 2.2M 2.2M 0.0k nvme8n15 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 244.4k 219.6k 0.0k 2.4M 2.1M 0.0k nvme8n16 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 225.6k 200.8k 0.0k 2.2M 2.0M 0.0k nvme8n17 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 203.6k 244.0k 0.0k 2.0M 2.4M 0.0k nvme8n18 2024-05-21T12:32:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 190.0k 280.8k 0.0k 1.9M 2.7M 0.0k nvme8n19 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 228.0k 219.2k 0.0k 2.2M 2.1M 0.0k nvme8n2 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 289.2k 173.2k 0.0k 2.8M 1.7M 0.0k nvme8n20 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.80 236.4k 176.8k 0.0k 2.3M 1.7M 0.0k nvme8n3 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 240.0k 233.6k 0.0k 2.3M 2.3M 0.0k nvme8n4 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 212.0k 220.0k 0.0k 2.1M 2.1M 0.0k nvme8n5 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 255.2k 216.4k 0.0k 2.5M 2.1M 0.0k nvme8n6 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 284.4k 181.6k 0.0k 2.8M 1.8M 0.0k nvme8n7 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 217.2k 215.2k 0.0k 2.1M 2.1M 0.0k nvme8n8 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.70 239.6k 208.8k 0.0k 2.3M 2.0M 0.0k nvme8n9 2024-05-21T12:32:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:24.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:24 smithi072 ceph-mon[33912]: pgmap v495: 33 pgs: 33 active+clean; 976 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 9.8 MiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-05-21T12:32:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:26 smithi092 ceph-mon[33665]: pgmap v496: 33 pgs: 33 active+clean; 983 MiB data, 823 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-05-21T12:32:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:26 smithi045 ceph-mon[30165]: pgmap v496: 33 pgs: 33 active+clean; 983 MiB data, 823 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-05-21T12:32:26.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:26 smithi072 ceph-mon[33912]: pgmap v496: 33 pgs: 33 active+clean; 983 MiB data, 823 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-05-21T12:32:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:28 smithi092 ceph-mon[33665]: pgmap v497: 33 pgs: 33 active+clean; 991 MiB data, 876 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 961 op/s 2024-05-21T12:32:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:28 smithi045 ceph-mon[30165]: pgmap v497: 33 pgs: 33 active+clean; 991 MiB data, 876 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 961 op/s 2024-05-21T12:32:28.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:28 smithi072 ceph-mon[33912]: pgmap v497: 33 pgs: 33 active+clean; 991 MiB data, 876 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 961 op/s 2024-05-21T12:32:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:30 smithi092 ceph-mon[33665]: pgmap v498: 33 pgs: 33 active+clean; 996 MiB data, 950 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 953 op/s 2024-05-21T12:32:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:30 smithi045 ceph-mon[30165]: pgmap v498: 33 pgs: 33 active+clean; 996 MiB data, 950 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 953 op/s 2024-05-21T12:32:30.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:30 smithi072 ceph-mon[33912]: pgmap v498: 33 pgs: 33 active+clean; 996 MiB data, 950 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 13 MiB/s wr, 953 op/s 2024-05-21T12:32:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:32 smithi092 ceph-mon[33665]: pgmap v499: 33 pgs: 33 active+clean; 998 MiB data, 999 MiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 12 MiB/s wr, 880 op/s 2024-05-21T12:32:32.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:32 smithi045 ceph-mon[30165]: pgmap v499: 33 pgs: 33 active+clean; 998 MiB data, 999 MiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 12 MiB/s wr, 880 op/s 2024-05-21T12:32:32.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:32 smithi072 ceph-mon[33912]: pgmap v499: 33 pgs: 33 active+clean; 998 MiB data, 999 MiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 12 MiB/s wr, 880 op/s 2024-05-21T12:32:34.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:34 smithi045 ceph-mon[30165]: pgmap v500: 33 pgs: 33 active+clean; 1003 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-21T12:32:34.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:34 smithi072 ceph-mon[33912]: pgmap v500: 33 pgs: 33 active+clean; 1003 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-21T12:32:34.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:34 smithi092 ceph-mon[33665]: pgmap v500: 33 pgs: 33 active+clean; 1003 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-21T12:32:34.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:32:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 191.2k 184.4k 0.0k 1.9M 1.8M 0.0k nvme2n1 2024-05-21T12:32:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 165.6k 196.4k 0.0k 1.6M 1.9M 0.0k nvme2n10 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 231.2k 122.8k 0.0k 2.3M 1.2M 0.0k nvme2n11 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 185.6k 190.0k 0.0k 1.8M 1.9M 0.0k nvme2n12 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 186.8k 178.0k 0.0k 1.8M 1.7M 0.0k nvme2n13 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 227.6k 154.0k 0.0k 2.2M 1.5M 0.0k nvme2n14 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 166.8k 189.6k 0.0k 1.6M 1.9M 0.0k nvme2n15 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 175.6k 174.0k 0.0k 1.7M 1.7M 0.0k nvme2n16 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 193.6k 161.6k 0.0k 1.9M 1.6M 0.0k nvme2n17 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 212.0k 180.4k 0.0k 2.1M 1.8M 0.0k nvme2n18 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 162.8k 150.0k 0.0k 1.6M 1.5M 0.0k nvme2n19 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 247.6k 176.0k 0.0k 2.4M 1.7M 0.0k nvme2n2 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 199.2k 177.2k 0.0k 1.9M 1.7M 0.0k nvme2n20 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 158.4k 175.2k 0.0k 1.5M 1.7M 0.0k nvme2n3 2024-05-21T12:32:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 174.0k 188.0k 0.0k 1.7M 1.8M 0.0k nvme2n4 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 211.6k 167.2k 0.0k 2.1M 1.6M 0.0k nvme2n5 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 212.4k 160.8k 0.0k 2.1M 1.6M 0.0k nvme2n6 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 163.6k 207.6k 0.0k 1.6M 2.0M 0.0k nvme2n7 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 209.2k 198.8k 0.0k 2.0M 1.9M 0.0k nvme2n8 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 216.8k 170.8k 0.0k 2.1M 1.7M 0.0k nvme2n9 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 184.0k 204.4k 0.0k 1.8M 2.0M 0.0k nvme5n1 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 157.6k 217.6k 0.0k 1.5M 2.1M 0.0k nvme5n10 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 175.2k 196.8k 0.0k 1.7M 1.9M 0.0k nvme5n11 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 146.8k 183.2k 0.0k 1.4M 1.8M 0.0k nvme5n12 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 160.4k 208.8k 0.0k 1.6M 2.0M 0.0k nvme5n13 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 186.0k 186.4k 0.0k 1.8M 1.8M 0.0k nvme5n14 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 186.8k 205.6k 0.0k 1.8M 2.0M 0.0k nvme5n15 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 172.4k 209.2k 0.0k 1.7M 2.0M 0.0k nvme5n16 2024-05-21T12:32:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 212.4k 177.2k 0.0k 2.1M 1.7M 0.0k nvme5n17 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 183.2k 189.6k 0.0k 1.8M 1.9M 0.0k nvme5n18 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 178.4k 206.4k 0.0k 1.7M 2.0M 0.0k nvme5n19 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 166.4k 221.6k 0.0k 1.6M 2.2M 0.0k nvme5n2 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 186.4k 186.8k 0.0k 1.8M 1.8M 0.0k nvme5n20 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 204.4k 195.2k 0.0k 2.0M 1.9M 0.0k nvme5n3 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 193.2k 142.4k 0.0k 1.9M 1.4M 0.0k nvme5n4 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 174.8k 199.2k 0.0k 1.7M 1.9M 0.0k nvme5n5 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 161.2k 222.4k 0.0k 1.6M 2.2M 0.0k nvme5n6 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 157.2k 193.2k 0.0k 1.5M 1.9M 0.0k nvme5n7 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 181.6k 190.8k 0.0k 1.8M 1.9M 0.0k nvme5n8 2024-05-21T12:32:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 171.2k 235.2k 0.0k 1.7M 2.3M 0.0k nvme5n9 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 198.0k 136.8k 0.0k 1.9M 1.3M 0.0k nvme8n1 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 195.6k 192.0k 0.0k 1.9M 1.9M 0.0k nvme8n10 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 160.0k 221.6k 0.0k 1.6M 2.2M 0.0k nvme8n11 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 176.4k 201.6k 0.0k 1.7M 2.0M 0.0k nvme8n12 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 191.6k 187.6k 0.0k 1.9M 1.8M 0.0k nvme8n13 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 172.8k 182.4k 0.0k 1.7M 1.8M 0.0k nvme8n14 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 182.8k 199.6k 0.0k 1.8M 1.9M 0.0k nvme8n15 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 203.6k 184.8k 0.0k 2.0M 1.8M 0.0k nvme8n16 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 198.4k 173.6k 0.0k 1.9M 1.7M 0.0k nvme8n17 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 195.6k 176.4k 0.0k 1.9M 1.7M 0.0k nvme8n18 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 198.8k 166.0k 0.0k 1.9M 1.6M 0.0k nvme8n19 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 185.2k 208.0k 0.0k 1.8M 2.0M 0.0k nvme8n2 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 150.8k 194.8k 0.0k 1.5M 1.9M 0.0k nvme8n20 2024-05-21T12:32:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 15.30 177.2k 198.4k 0.0k 1.7M 1.9M 0.0k nvme8n3 2024-05-21T12:32:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 165.6k 224.8k 0.0k 1.6M 2.2M 0.0k nvme8n4 2024-05-21T12:32:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 217.6k 175.6k 0.0k 2.1M 1.7M 0.0k nvme8n5 2024-05-21T12:32:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 160.8k 202.8k 0.0k 1.6M 2.0M 0.0k nvme8n6 2024-05-21T12:32:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 214.8k 199.6k 0.0k 2.1M 1.9M 0.0k nvme8n7 2024-05-21T12:32:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 216.4k 169.6k 0.0k 2.1M 1.7M 0.0k nvme8n8 2024-05-21T12:32:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 15.40 203.2k 162.4k 0.0k 2.0M 1.6M 0.0k nvme8n9 2024-05-21T12:32:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:36.355 INFO:tasks.nvmeof.[nvmeof.thrasher]:done waiting before reviving 2024-05-21T12:32:36.355 INFO:tasks.nvmeof.[nvmeof.thrasher]:display and verify stats: 2024-05-21T12:32:36.356 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:32:36.392 INFO:teuthology.orchestra.run.smithi045.stdout:× ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service - Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:32:36.392 INFO:teuthology.orchestra.run.smithi045.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:32:36.392 INFO:teuthology.orchestra.run.smithi045.stdout: Active: failed (Result: exit-code) since Tue 2024-05-21 12:30:34 UTC; 2min 2s ago 2024-05-21T12:32:36.392 INFO:teuthology.orchestra.run.smithi045.stdout: Duration: 5min 52.904s 2024-05-21T12:32:36.392 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 62788 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 62789 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 72363 ExecStop=/bin/bash -c bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.stop (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 72433 ExecStopPost=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.poststop (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 72463 ExecStopPost=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout: Main PID: 63408 (code=exited, status=137) 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout: CPU: 6min 21.557s 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:23 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[63408]: [21-May-2024 12:30:23] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log.gz 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:33 smithi045 bash[72391]: time="2024-05-21T12:30:33Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a in 10 seconds, resorting to SIGKILL" 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:33 smithi045 podman[72391]: 2024-05-21 12:30:33.264117934 +0000 UTC m=+10.123597432 container died 4d34dcd5cab0f2eb889b7fac4e0f0a4afe77f19e67be6a91c8b786a42e67b4d5 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.version=24.01, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.branch=undefined, io.ceph.summary=Ceph NVMe over Fabrics Gateway, build-date=2024-05-19 17:57:19 UTC, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.git.repo=https://github.com/ceph/spdk.git, com.redhat.component=ubi9-container, vendor=Red Hat, Inc., io.ceph.git.branch=tags/1.2.9, io.ceph.version=1.2.9, release=, io.ceph.component=ceph-nvmeof, url=https://github.com/ceph/ceph-nvmeof, io.ceph.nvmeof=None, io.spdk.build.ceph-release=19.0.0, io.spdk.build.pkgdep-args=--rbd, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.target-arch=, org.centos.version=9.0-21.el9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, distribution-scope=public, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.makeflags=, vcs-type=git, architecture=x86_64, io.openshift.wants=ceph,rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.buildah.version=1.29.0, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, version=1.2.9, io.ceph.url=https://github.com/ceph/ceph-nvmeof, summary=Ceph NVMe over Fabrics Gateway, io.ceph.maintainer=Ceph Developers ) 2024-05-21T12:32:36.393 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:33 smithi045 podman[72391]: 2024-05-21 12:30:33.696774781 +0000 UTC m=+10.556254260 container cleanup 4d34dcd5cab0f2eb889b7fac4e0f0a4afe77f19e67be6a91c8b786a42e67b4d5 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.ceph-release=19.0.0, io.spdk.build.target-arch=, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.buildah.version=1.29.0, org.centos.version=9.0-21.el9, release=, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, version=1.2.9, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, com.redhat.component=ubi9-container, name=ceph-nvmeof, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, summary=Ceph NVMe over Fabrics Gateway, io.ceph.version=1.2.9, io.spdk.build.pkgdep-args=--rbd, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.spdk.build.makeflags=, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vendor=Red Hat, Inc., io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, url=https://github.com/ceph/ceph-nvmeof, io.ceph.summary=Ceph NVMe over Fabrics Gateway, distribution-scope=public, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.branch=undefined, io.openshift.wants=ceph,rbd, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., architecture=x86_64, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.component=ceph-nvmeof, io.ceph.maintainer=Ceph Developers , vcs-type=git, build-date=2024-05-19 17:57:19 UTC, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.nvmeof=None, io.spdk.version=24.01, io.ceph.git.branch=tags/1.2.9, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-05-21T12:32:36.394 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:33 smithi045 bash[72391]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a 2024-05-21T12:32:36.394 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:34 smithi045 podman[72421]: 2024-05-21 12:30:34.106971247 +0000 UTC m=+0.835779331 container remove 4d34dcd5cab0f2eb889b7fac4e0f0a4afe77f19e67be6a91c8b786a42e67b4d5 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, version=1.2.9, maintainer=Ceph Developers , io.spdk.version=24.01, io.buildah.version=1.29.0, release=, url=https://github.com/ceph/ceph-nvmeof, io.openshift.wants=ceph,rbd, io.spdk.git.repo=https://github.com/ceph/spdk.git, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.url=https://github.com/ceph/ceph-nvmeof, com.redhat.component=ubi9-container, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., distribution-scope=public, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, name=ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, architecture=x86_64, io.spdk.git.branch=undefined, io.spdk.build.makeflags=, io.spdk.build.pkgdep-args=--rbd, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.ceph-release=19.0.0, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.nvmeof=None, io.ceph.maintainer=Ceph Developers , summary=Ceph NVMe over Fabrics Gateway, vcs-type=git, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, org.centos.version=9.0-21.el9, io.ceph.component=ceph-nvmeof, io.spdk.build.target-arch=, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.ceph.version=1.2.9, build-date=2024-05-19 17:57:19 UTC, vendor=Red Hat, Inc., io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF).) 2024-05-21T12:32:36.394 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:34 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:32:36.394 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:34 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Failed with result 'exit-code'. 2024-05-21T12:32:36.394 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:34 smithi045 systemd[1]: Stopped Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:32:36.394 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:30:34 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Consumed 6min 21.557s CPU time. 2024-05-21T12:32:36.395 DEBUG:teuthology.orchestra.run:got remote process result: 3 2024-05-21T12:32:36.395 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:32:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service - Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:32:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:32:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: Active: active (running) since Tue 2024-05-21 12:24:10 UTC; 8min ago 2024-05-21T12:32:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 53097 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 53098 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: Main PID: 53263 (conmon) 2024-05-21T12:32:36.431 INFO:teuthology.orchestra.run.smithi072.stdout: Tasks: 126 (limit: 202814) 2024-05-21T12:32:36.431 INFO:teuthology.orchestra.run.smithi072.stdout: Memory: 1.0G 2024-05-21T12:32:36.431 INFO:teuthology.orchestra.run.smithi072.stdout: CPU: 8min 23.464s 2024-05-21T12:32:36.431 INFO:teuthology.orchestra.run.smithi072.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:32:36.431 INFO:teuthology.orchestra.run.smithi072.stdout: ├─libpod-payload-a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 2024-05-21T12:32:36.431 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─53265 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:32:36.432 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─53267 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:32:36.432 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─53289 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.b --gateway-address 172.21.15.72:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.72:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.b -k /etc/ceph/keyring 2024-05-21T12:32:36.432 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─53805 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:32:36.432 INFO:teuthology.orchestra.run.smithi072.stdout: │ └─53807 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:32:36.433 INFO:teuthology.orchestra.run.smithi072.stdout: └─runtime 2024-05-21T12:32:36.433 INFO:teuthology.orchestra.run.smithi072.stdout: └─53263 /usr/bin/conmon --api-version 1 -c a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 -u a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243/userdata -p /run/containers/storage/overlay-containers/a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 2024-05-21T12:32:36.433 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:32:36.433 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:32:36.433 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:32:36.433 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:32:36.433 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:32:36.433 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:32:36.434 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:32:36.434 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: OPTIMIZED 2024-05-21T12:32:36.434 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:32:36.434 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: ], , client address: IPv4 172.21.15.72:57980 2024-05-21T12:32:36.434 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:30:45 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [2024-05-21 12:30:45.012275] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000267 seconds 2024-05-21T12:32:36.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:32:36.471 INFO:teuthology.orchestra.run.smithi092.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service - Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:32:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:32:36.471 INFO:teuthology.orchestra.run.smithi092.stdout: Active: active (running) since Tue 2024-05-21 12:23:56 UTC; 8min ago 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53502 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53503 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: Main PID: 53679 (conmon) 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: Tasks: 126 (limit: 202816) 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: Memory: 1.0G 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: CPU: 8min 22.068s 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: ├─libpod-payload-5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53681 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53683 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53701 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.c --gateway-address 172.21.15.92:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.92:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.c -k /etc/ceph/keyring 2024-05-21T12:32:36.472 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─54233 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: │ └─54267 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: └─runtime 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: └─53679 /usr/bin/conmon --api-version 1 -c 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d -u 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata -p /run/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:53.769517] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:32:36.473 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:55.476877] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:32:36.474 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:32:36.474 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:30:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:30:00] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:52412 2024-05-21T12:32:36.475 DEBUG:teuthology.orchestra.run.smithi137:> ceph orch ls && ceph orch ps --daemon-type nvmeof && ceph health detail && ceph -s && sudo nvme list-subsys /dev/nvme5n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n20 | grep 'live optimized' 2024-05-21T12:32:36.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:36 smithi072 ceph-mon[33912]: pgmap v501: 33 pgs: 33 active+clean; 1004 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 928 op/s 2024-05-21T12:32:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:36 smithi092 ceph-mon[33665]: pgmap v501: 33 pgs: 33 active+clean; 1004 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 928 op/s 2024-05-21T12:32:36.799 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T12:32:36.799 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 1/1 8m ago 13m smithi045=x;count:1 2024-05-21T12:32:36.799 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 8m ago 14m smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:32:36.799 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.mypool ?:4420,5500,8009 3/3 8m ago 8m smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:32:36.799 INFO:teuthology.orchestra.run.smithi137.stdout:osd 8 8m ago - 2024-05-21T12:32:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:36 smithi045 ceph-mon[30165]: pgmap v501: 33 pgs: 33 active+clean; 1004 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 928 op/s 2024-05-21T12:32:37.104 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:32:37.104 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.a smithi045 *:5500,4420,8009 running (8m) 8m ago 8m 47.9M - 1.2.9 30dd525ace30 4d34dcd5cab0 2024-05-21T12:32:37.104 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.b smithi072 *:5500,4420,8009 running (8m) 8m ago 8m 44.6M - 1.2.9 30dd525ace30 a048ad7abf63 2024-05-21T12:32:37.104 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.c smithi092 *:5500,4420,8009 running (8m) 8m ago 8m 45.1M - 1.2.9 30dd525ace30 5e5a8ffd2ba6 2024-05-21T12:32:37.463 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_OK 2024-05-21T12:32:37.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:37 smithi072 ceph-mon[33912]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:32:37.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:37 smithi092 ceph-mon[33665]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: cluster: 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: health: HEALTH_OK 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: services: 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: mon: 3 daemons, quorum a,c,b (age 14m) 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: mgr: x(active, since 15m) 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-05-21T12:32:37.812 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: data: 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: pools: 2 pools, 33 pgs 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: objects: 485 objects, 1007 MiB 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: usage: 1.2 GiB used, 714 GiB / 715 GiB avail 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: pgs: 33 active+clean 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: io: 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: client: 11 MiB/s rd, 11 MiB/s wr, 481 op/s rd, 470 op/s wr 2024-05-21T12:32:37.813 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:32:37.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:37 smithi045 ceph-mon[30165]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:32:37.844 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:37.863 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:37.882 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:37.901 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:37.920 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:37.939 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:37.958 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:37.976 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:37.995 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.014 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.032 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.050 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.068 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.086 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.105 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.123 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.142 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.159 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.178 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.196 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.214 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.233 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.252 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.270 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.289 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.307 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.326 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.344 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.362 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.381 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.400 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.419 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.438 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.456 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.475 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.494 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.513 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.531 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.550 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.568 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.587 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.605 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.624 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.642 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.660 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:38 smithi072 ceph-mon[33912]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:32:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:38 smithi072 ceph-mon[33912]: pgmap v502: 33 pgs: 33 active+clean; 1007 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 951 op/s 2024-05-21T12:32:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:38 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/2876354111' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:32:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:38 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/459453784' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:32:38.678 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.697 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.715 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.733 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.751 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:38 smithi092 ceph-mon[33665]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:32:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:38 smithi092 ceph-mon[33665]: pgmap v502: 33 pgs: 33 active+clean; 1007 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 951 op/s 2024-05-21T12:32:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:38 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/2876354111' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:32:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:38 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/459453784' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:32:38.769 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.788 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.806 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:38 smithi045 ceph-mon[30165]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:32:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:38 smithi045 ceph-mon[30165]: pgmap v502: 33 pgs: 33 active+clean; 1007 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 951 op/s 2024-05-21T12:32:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:38 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/2876354111' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:32:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:38 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/459453784' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:32:38.824 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.842 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.860 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.879 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.898 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.917 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.936 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:32:38.936 INFO:tasks.nvmeof.[nvmeof.thrasher]:reviving nvmeof.a 2024-05-21T12:32:38.936 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Restarting nvmeof.nvmeof.a (starting--it wasn't running)... 2024-05-21T12:32:38.936 DEBUG:teuthology.orchestra.run.smithi045:nvmeof.nvmeof.a> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:32:38.939 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl start ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:32:39.249 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 systemd[1]: Starting Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:32:39.511 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 podman[72870]: 2024-05-21 12:32:39.152850705 +0000 UTC m=+0.019554193 image pull 30dd525ace30bc703cdf4a290b3be2b24f6ca7b74757424a10799eaf370636b4 quay.io/ceph/nvmeof:1.2 2024-05-21T12:32:39.511 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 podman[72870]: 2024-05-21 12:32:39.256878822 +0000 UTC m=+0.123582293 container create c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.branch=tags/1.2.9, io.openshift.wants=ceph,rbd, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.buildah.version=1.29.0, io.ceph.component=ceph-nvmeof, com.redhat.component=ubi9-container, url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.makeflags=, version=1.2.9, org.centos.version=9.0-21.el9, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.nvmeof=None, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.ceph-release=19.0.0, release=, io.spdk.build.target-arch=, maintainer=Ceph Developers , io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.branch=undefined, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.maintainer=Ceph Developers , description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, summary=Ceph NVMe over Fabrics Gateway, distribution-scope=public, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, build-date=2024-05-19 17:57:19 UTC, architecture=x86_64, io.spdk.build.pkgdep-args=--rbd, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.version=24.01, vendor=Red Hat, Inc., io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.version=1.2.9, vcs-type=git, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, name=ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec) 2024-05-21T12:32:39.511 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 podman[72870]: 2024-05-21 12:32:39.509808757 +0000 UTC m=+0.376512227 container init c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, com.redhat.component=ubi9-container, name=ceph-nvmeof, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.version=1.2.9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, url=https://github.com/ceph/ceph-nvmeof, maintainer=Ceph Developers , org.centos.version=9.0-21.el9, architecture=x86_64, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vendor=Red Hat, Inc., io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.version=24.01, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.target-arch=, io.spdk.build.pkgdep-args=--rbd, io.spdk.build.makeflags=, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.nvmeof=None, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.branch=undefined, release=, build-date=2024-05-19 17:57:19 UTC, distribution-scope=public, version=1.2.9, io.ceph.git.branch=tags/1.2.9, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.component=ceph-nvmeof, io.buildah.version=1.29.0, summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.ceph-release=19.0.0, io.openshift.wants=ceph,rbd, vcs-type=git, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.maintainer=Ceph Developers ) 2024-05-21T12:32:39.607 INFO:tasks.nvmeof.[nvmeof.thrasher]:waiting for 84 secs before thrashing 2024-05-21T12:32:39.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 podman[72870]: 2024-05-21 12:32:39.513721775 +0000 UTC m=+0.380425244 container start c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.wants=ceph,rbd, architecture=x86_64, io.ceph.component=ceph-nvmeof, io.buildah.version=1.29.0, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, build-date=2024-05-19 17:57:19 UTC, vendor=Red Hat, Inc., io.ceph.nvmeof=None, io.spdk.git.branch=undefined, release=, org.centos.version=9.0-21.el9, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.version=24.01, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.ceph-release=19.0.0, io.spdk.build.pkgdep-args=--rbd, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, distribution-scope=public, name=ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.target-arch=, summary=Ceph NVMe over Fabrics Gateway, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, vcs-type=git, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.makeflags=, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.version=1.2.9, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.ceph.maintainer=Ceph Developers , url=https://github.com/ceph/ceph-nvmeof, version=1.2.9, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.url=https://github.com/ceph/ceph-nvmeof, maintainer=Ceph Developers , io.ceph.git.branch=tags/1.2.9, com.redhat.component=ubi9-container) 2024-05-21T12:32:39.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 bash[72870]: c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db 2024-05-21T12:32:39.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 systemd[1]: Started Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:32:40.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO utils.py:258: Initialize gateway log level to "INFO" 2024-05-21T12:32:40.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO utils.py:271: Log files will be saved in /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a, using rotation 2024-05-21T12:32:40.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:78: Using NVMeoF gateway version 1.2.9 2024-05-21T12:32:40.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:81: Using SPDK version 24.01 2024-05-21T12:32:40.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:84: Using vstart cluster version based on 19.0.0 2024-05-21T12:32:40.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:87: NVMeoF gateway built on: 2024-05-19 17:57:19 UTC 2024-05-21T12:32:40.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:90: NVMeoF gateway Git repository: https://github.com/ceph/ceph-nvmeof 2024-05-21T12:32:40.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:93: NVMeoF gateway Git branch: tags/1.2.9 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:96: NVMeoF gateway Git commit: 06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:102: SPDK Git repository: https://github.com/ceph/spdk.git 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:105: SPDK Git branch: undefined 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:108: SPDK Git commit: dade5b4c96a647e63ef96d9a5e05f3b64ea0598a 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:59: Using configuration file /src/ceph-nvmeof.conf 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:61: ====================================== Configuration file content ====================================== 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: # This file is generated by cephadm. 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: [gateway] 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: name = client.nvmeof.nvmeof.a 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: group = 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: addr = 172.21.15.45 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: port = 5500 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: enable_auth = False 2024-05-21T12:32:40.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: state_update_notify = True 2024-05-21T12:32:40.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: state_update_interval_sec = 5 2024-05-21T12:32:40.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: enable_spdk_discovery_controller = False 2024-05-21T12:32:40.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: enable_prometheus_exporter = True 2024-05-21T12:32:40.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: prometheus_exporter_ssl = False 2024-05-21T12:32:40.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: prometheus_port = 10008 2024-05-21T12:32:40.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: verify_nqns = True 2024-05-21T12:32:40.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: omap_file_lock_duration = 20 2024-05-21T12:32:40.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: omap_file_lock_retries = 30 2024-05-21T12:32:40.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: omap_file_lock_retry_sleep_interval = 1.0 2024-05-21T12:32:40.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: omap_file_update_reloads = 10 2024-05-21T12:32:40.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: allowed_consecutive_spdk_ping_failures = 1 2024-05-21T12:32:40.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: spdk_ping_interval_in_seconds = 2.0 2024-05-21T12:32:40.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: ping_spdk_under_lock = False 2024-05-21T12:32:40.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: enable_monitor_client = True 2024-05-21T12:32:40.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: 2024-05-21T12:32:40.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: [gateway-logs] 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: log_level = INFO 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: log_files_enabled = True 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: log_files_rotation_enabled = True 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: verbose_log_messages = True 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: max_log_file_size_in_mb = 10 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: max_log_files_count = 20 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: max_log_directory_backups = 10 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: log_directory = /var/log/ceph/ 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: [discovery] 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: addr = 172.21.15.45 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: port = 8009 2024-05-21T12:32:40.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: 2024-05-21T12:32:40.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: [ceph] 2024-05-21T12:32:40.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: pool = mypool 2024-05-21T12:32:40.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: config_file = /etc/ceph/ceph.conf 2024-05-21T12:32:40.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: id = nvmeof.nvmeof.a 2024-05-21T12:32:40.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: 2024-05-21T12:32:40.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: [mtls] 2024-05-21T12:32:40.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: server_key = ./server.key 2024-05-21T12:32:40.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: client_key = ./client.key 2024-05-21T12:32:40.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: server_cert = ./server.crt 2024-05-21T12:32:40.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: client_cert = ./client.crt 2024-05-21T12:32:40.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: 2024-05-21T12:32:40.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: [spdk] 2024-05-21T12:32:40.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: tgt_path = /usr/local/bin/nvmf_tgt 2024-05-21T12:32:40.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: rpc_socket_dir = /var/tmp/ 2024-05-21T12:32:40.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: rpc_socket_name = spdk.sock 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: timeout = 60.0 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: bdevs_per_cluster = 32 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: log_level = WARNING 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: conn_retries = 10 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: transports = tcp 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: transport_tcp_options = {"in_capsule_data_size": 8192, "max_io_qpairs_per_ctrlr": 7} 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: [monitor] 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:65: timeout = 1.0 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO config.py:66: ======================================================================================================== 2024-05-21T12:32:40.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:90: Starting gateway client.nvmeof.nvmeof.a 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:157: Starting serve, monitor client version: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:236: Starting /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.a --gateway-address 172.21.15.45:5500 --gateway-pool mypool --gateway-group --monitor-group-address 172.21.15.45:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.a -k /etc/ceph/keyring 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:240: monitor client process id: 19 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:146: MonitorGroup server is listening on 172.21.15.45:5499 for group id 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.857+0000 7ff1706095c0 0 nvmeofgw int NVMeofGwMonitorClient::init() 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.857+0000 7ff1706095c0 0 nvmeofgw int NVMeofGwMonitorClient::init() gateway name: client.nvmeof.nvmeof.a pool:mypool group: address: 172.21.15.45:5500 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.857+0000 7ff1706095c0 0 nvmeofgw int NVMeofGwMonitorClient::init() nvmeof Registered monc callback 2024-05-21T12:32:40.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.859+0000 7ff1706095c0 0 nvmeofgw int NVMeofGwMonitorClient::init() monc.authentication done 2024-05-21T12:32:40.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.860+0000 7ff1706095c0 0 nvmeofgw void NVMeofGwMonitorClient::tick() 2024-05-21T12:32:40.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.861+0000 7ff1706095c0 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:32:40.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.861+0000 7ff1706095c0 0 nvmeofgw void NVMeofGwMonitorClient::send_beacon() sending beacon as gid 15204 availability 0 osdmap_epoch 0 gwmap_epoch 0 2024-05-21T12:32:40.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.861+0000 7ff1706095c0 0 nvmeofgw int NVMeofGwMonitorClient::init() Complete. 2024-05-21T12:32:40.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.861+0000 7ff16c5a6640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:32:40.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.861+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:32:40.051 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 119 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 119 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.861+0000 7ff16c5a6640 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.861+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_group_id: 0 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:135: Gateway client.nvmeof.nvmeof.a group id=0 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:149: Stopping the MonitorGroup server... 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.052 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.053 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.054 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.055 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.056 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.863+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 119 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.864+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.866+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:152: The MonitorGroup gRPC server stopped... 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:349: SPDK Target Path: /usr/local/bin/nvmf_tgt 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.867+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:351: SPDK Socket: /var/tmp/spdk.sock 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:357: Starting /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.869+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.057 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:370: SPDK process id: 67 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO server.py:371: Attempting to initialize SPDK: rpc_socket: /var/tmp/spdk.sock, conn_retries: 300, timeout: 60.0 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: INFO: Setting log level to WARNING 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:39] INFO client.py:110: Setting log level to WARNING 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.870+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.872+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.873+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.875+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.876+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.878+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.880+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.058 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.881+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.883+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:39.884349] Starting SPDK v24.01 / DPDK 23.11.0 initialization... 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:39.884610] [ DPDK EAL parameters: nvmf --no-shconf -c 0x1 --no-pci --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --iova-mode=pa --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid67 ] 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.884+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.886+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.887+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.889+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.890+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.892+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.893+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.895+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.896+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.059 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.898+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.899+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.901+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.903+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.904+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.905+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.907+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.908+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.910+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.911+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.913+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.914+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.916+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.060 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.917+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.919+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.920+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.922+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.923+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.925+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.927+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.928+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.930+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.931+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.933+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.934+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.936+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.061 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.937+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.939+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.940+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.942+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.943+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.945+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.947+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.948+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.950+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.951+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.953+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.954+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.062 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.956+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.957+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.959+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.960+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.962+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.963+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.965+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.966+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.968+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.970+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.971+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.973+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.974+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.976+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.063 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.978+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.979+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.981+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.982+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.984+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.985+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.987+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.988+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.990+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.992+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.993+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.995+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.996+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.998+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:39.999766] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:39 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:39.999+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.001+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.003+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.005+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.006+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.008+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.010+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.011+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.013+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.014+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.016+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.017+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.019+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.021+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.022+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.024+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.026+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.028+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.029+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.031+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.032+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.034+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.035+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.037+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.038+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.040+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.042+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.043+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.282 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.045+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.282 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.046+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.282 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.048+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.282 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.050+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.282 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.051+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.053+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.054806] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.055+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.057+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.058+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.060+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.062+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.064+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.283 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.066+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.068+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.069+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.071+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.072+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.074+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.075+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.077+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.079+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.081+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.083+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.284 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.085+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.285 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.087+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.285 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.088+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.285 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.090+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.285 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.092+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.285 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.094+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.285 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.095+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.285 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.097+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.285 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.098+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.286 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.100+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.102+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.103+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.105+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.106+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.108+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.109+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.111+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.112+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.114+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.115+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.287 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.117+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.118+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.120+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.121+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.123+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.124+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.126+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.127+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.129+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.130+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.132+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.133+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.135+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.288 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.136+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.138+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.139+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.141+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.142+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.144+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.145+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.147+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.148+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.150+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.151+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.153+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.154+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.289 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.156+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.157+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.159+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.161+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.162+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.164+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.165+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.167+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.168+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.290 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.170+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.291 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.171+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.291 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.173+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.291 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.174+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.291 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.176+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.291 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.177+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.291 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.179+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.180+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.182+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.183+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.185+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.186+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.188+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.189+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.191+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.192+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.194+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.196+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.197+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.199+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.200+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.201+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.203+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.204+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.206+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.207+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.209+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.211+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.212+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.214+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.215+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.217+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.218+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.220+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.221+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.223+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.224+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.226+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.227+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.229+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.230+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.232+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.298 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.233+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.298 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.235+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.298 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.236+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.298 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.238+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.298 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.239+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.298 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.241+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.299 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.243+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.299 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.244+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.299 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.246+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.299 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.247+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.299 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.249+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.299 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.250+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.299 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.252+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.253+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.255+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.256+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.258+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.259+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.261+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.263+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.264+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.266+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.267+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.300 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.269+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.271+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.272+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.273686] tcp.c: 659:nvmf_tcp_create: *NOTICE*: *** TCP Transport Init *** 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.274+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.275+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.277+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.278+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.280+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.281+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.283+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.301 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.285+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.558 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-mon[30165]: pgmap v503: 33 pgs: 33 active+clean; 1009 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 933 op/s 2024-05-21T12:32:40.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3869454575' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:32:40.559 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.286+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.559 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO server.py:268: Discovery service process id: 870 2024-05-21T12:32:40.559 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO server.py:264: Starting ceph nvmeof discovery service 2024-05-21T12:32:40.559 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.288+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.559 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.290+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.559 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.291+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.559 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.293+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.295+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.296+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.298+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.300+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.302+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.304+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO discovery.py:322: log pages info from omap: nvmeof.state 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO cephutils.py:74: Connected to Ceph with version "19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)" 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:86: Requested huge pages count is 2048 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:105: Actual huge pages count is 4096 2024-05-21T12:32:40.560 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:190: NVMeoF bdevs per cluster: 32 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.305+0000 7ff16c5a6640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO server.py:195: Prometheus endpoint is enabled 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO prometheus.py:98: Prometheus exporter running in http mode, listening on port 10008 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO discovery.py:330: discovery addr: 172.21.15.45 port: 8009 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 1 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.561 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 2 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 3 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 4 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 5 2024-05-21T12:32:40.562 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.563 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.563 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.563 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 6 2024-05-21T12:32:40.563 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.563 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.563 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.563 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 7 2024-05-21T12:32:40.564 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.564 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.564 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.564 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 8 2024-05-21T12:32:40.564 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 9 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 10 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 11 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.565 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 12 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 13 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 14 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 15 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 16 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 1 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 2 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 3 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 4 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 5 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 6 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 7 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 8 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 9 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 10 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 11 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 12 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 13 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 14 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 15 2024-05-21T12:32:40.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 16 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 1 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 2 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 3 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 4 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 5 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 6 2024-05-21T12:32:40.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 7 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 8 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 9 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 10 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 11 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 12 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 13 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 14 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 15 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 16 2024-05-21T12:32:40.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: INACCESSIBLE 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: ], , client address: IPv4 172.21.15.45:42360 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode1 not found 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode3 not found 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:2648: Received request to get gateway's info 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.318+0000 7ff16c5a6640 0 ms_deliver_dispatch: unhandled message 0x55fa58f45180 nvmeofgwmap magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO prometheus.py:131: Stats for all bdevs will be provided 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.319+0000 7ff16c5a6640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 4 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.319+0000 7ff16c5a6640 0 ms_deliver_dispatch: unhandled message 0x55fa58f35d40 mon_map magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: 2024-05-21T12:32:40.319+0000 7ff16c5a6640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 41 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:32:40.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.362451] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.45:0/876760989, rc 0 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.362558] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.45:0/876760989 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.45:0/876760989' anagrp=2 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.375717] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.396576] nvmf_rpc.c: 275:rpc_nvmf_get_subsystems: *WARNING*: rpc_nvmf_get_subsystems: deprecated feature listener.transport is deprecated in favor of trtype to be removed in v24.05 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.414127] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.45:0/3363088059, rc 0 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.414242] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.45:0/3363088059 2024-05-21T12:32:40.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.45:0/3363088059' anagrp=1 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.426660] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.463579] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.45:0/1896073667, rc 0 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.463679] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.45:0/1896073667 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.45:0/1896073667' anagrp=3 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.477203] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da 2024-05-21T12:32:40.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:32:40.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:32:40.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:40 smithi072 ceph-mon[33912]: pgmap v503: 33 pgs: 33 active+clean; 1009 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 933 op/s 2024-05-21T12:32:40.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:40 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3869454575' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:32:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:40 smithi092 ceph-mon[33665]: pgmap v503: 33 pgs: 33 active+clean; 1009 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 933 op/s 2024-05-21T12:32:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:40 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3869454575' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:32:40.809 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.571522] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:32:40.809 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.593468] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.614944] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b 2024-05-21T12:32:40.810 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.635277] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.656527] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:32:40.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:32:40.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.676694] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:32:40.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 2024-05-21T12:32:40.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:32:40.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:32:40.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.696369] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:32:40.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.716539] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.738393] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:32:40.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.759116] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.780268] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:32:40.814 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.800353] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:32:40.815 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 2024-05-21T12:32:40.815 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:32:40.815 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:40.815 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.821538] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.843111] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:32:41.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.862834] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:32:41.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f 2024-05-21T12:32:41.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:32:41.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:32:41.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.882428] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.902533] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.923210] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:32:41.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.944675] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.964589] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:32:41.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:40.984487] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:40 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.004562] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:32:41.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.024771] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.045421] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.065862] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:32:41.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 2024-05-21T12:32:41.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.087733] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.107987] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:32:41.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:32:41.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.128503] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:32:41.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 2024-05-21T12:32:41.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.148432] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.168197] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:32:41.319 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.187673] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.208357] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.228099] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.320 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.248579] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.268263] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.288880] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:32:41.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:32:41.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.308748] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:32:41.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:32:41.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:32:41.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:32:41.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.328677] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:32:41.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 2024-05-21T12:32:41.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:32:41.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:32:41.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.349746] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.370889] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:32:41.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.391939] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:32:41.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b 2024-05-21T12:32:41.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:32:41.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:32:41.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.411556] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:32:41.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 2024-05-21T12:32:41.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.431973] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.452087] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:32:41.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.471630] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.491470] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.511735] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d 2024-05-21T12:32:41.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.532437] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.551965] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:32:41.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.572291] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:32:41.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed 2024-05-21T12:32:42.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:32:42.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:42.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:32:42.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.593117] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:32:42.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 2024-05-21T12:32:42.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:32:42.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:42.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.613181] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.633483] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.652877] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 2024-05-21T12:32:42.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.672601] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.692688] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:32:42.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.712478] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: None 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: None 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: None 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: None 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: None 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: None 2024-05-21T12:32:42.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: None 2024-05-21T12:32:42.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: None 2024-05-21T12:32:42.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:41.749095] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.45 port 4420 *** 2024-05-21T12:32:42.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: None 2024-05-21T12:32:42.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: None 2024-05-21T12:32:42.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: None 2024-05-21T12:32:42.071 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:41 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:41] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: None 2024-05-21T12:32:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:42 smithi092 ceph-mon[33665]: pgmap v504: 33 pgs: 33 active+clean; 1010 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.7 MiB/s wr, 847 op/s 2024-05-21T12:32:42.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:42 smithi045 ceph-mon[30165]: pgmap v504: 33 pgs: 33 active+clean; 1010 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.7 MiB/s wr, 847 op/s 2024-05-21T12:32:42.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-mon[33912]: pgmap v504: 33 pgs: 33 active+clean; 1010 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.7 MiB/s wr, 847 op/s 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:32:42] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:32:42.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:32:42.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:32:42.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:32:42.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:32:42.917 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: ], , client address: IPv4 172.21.15.72:59920 2024-05-21T12:32:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:44 smithi092 ceph-mon[33665]: pgmap v505: 33 pgs: 33 active+clean; 1012 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-05-21T12:32:44.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:44 smithi045 ceph-mon[30165]: pgmap v505: 33 pgs: 33 active+clean; 1012 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-05-21T12:32:44.865 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:44.872 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:32:44.872 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 146.8k 112.0k 0.0k 1.4M 1.1M 0.0k nvme2n1 2024-05-21T12:32:44.872 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 119.2k 124.8k 0.0k 1.2M 1.2M 0.0k nvme2n10 2024-05-21T12:32:44.872 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 135.6k 118.8k 0.0k 1.3M 1.2M 0.0k nvme2n11 2024-05-21T12:32:44.872 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 138.0k 107.2k 0.0k 1.3M 1.0M 0.0k nvme2n12 2024-05-21T12:32:44.872 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 140.4k 96.4k 0.0k 1.4M 964.0k 0.0k nvme2n13 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 118.0k 123.2k 0.0k 1.2M 1.2M 0.0k nvme2n14 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 96.0k 137.2k 0.0k 960.0k 1.3M 0.0k nvme2n15 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 109.2k 135.2k 0.0k 1.1M 1.3M 0.0k nvme2n16 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 139.2k 130.4k 0.0k 1.4M 1.3M 0.0k nvme2n17 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 128.8k 125.2k 0.0k 1.3M 1.2M 0.0k nvme2n18 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 134.4k 138.4k 0.0k 1.3M 1.4M 0.0k nvme2n19 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 152.8k 106.8k 0.0k 1.5M 1.0M 0.0k nvme2n2 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 154.8k 122.0k 0.0k 1.5M 1.2M 0.0k nvme2n20 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 134.8k 150.0k 0.0k 1.3M 1.5M 0.0k nvme2n3 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 128.0k 138.8k 0.0k 1.2M 1.4M 0.0k nvme2n4 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 125.2k 139.2k 0.0k 1.2M 1.4M 0.0k nvme2n5 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 122.4k 126.8k 0.0k 1.2M 1.2M 0.0k nvme2n6 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 131.6k 142.4k 0.0k 1.3M 1.4M 0.0k nvme2n7 2024-05-21T12:32:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 108.0k 106.0k 0.0k 1.1M 1.0M 0.0k nvme2n8 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 116.4k 145.6k 0.0k 1.1M 1.4M 0.0k nvme2n9 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 128.4k 124.0k 0.0k 1.3M 1.2M 0.0k nvme5n1 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 146.4k 111.2k 0.0k 1.4M 1.1M 0.0k nvme5n10 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 112.4k 157.2k 0.0k 1.1M 1.5M 0.0k nvme5n11 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 130.8k 134.0k 0.0k 1.3M 1.3M 0.0k nvme5n12 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 119.2k 142.0k 0.0k 1.2M 1.4M 0.0k nvme5n13 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 131.6k 139.6k 0.0k 1.3M 1.4M 0.0k nvme5n14 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 132.8k 113.2k 0.0k 1.3M 1.1M 0.0k nvme5n15 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 122.4k 146.4k 0.0k 1.2M 1.4M 0.0k nvme5n16 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 134.0k 122.8k 0.0k 1.3M 1.2M 0.0k nvme5n17 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 148.0k 122.0k 0.0k 1.4M 1.2M 0.0k nvme5n18 2024-05-21T12:32:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 108.8k 144.8k 0.0k 1.1M 1.4M 0.0k nvme5n19 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 127.6k 111.6k 0.0k 1.2M 1.1M 0.0k nvme5n2 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 123.6k 126.0k 0.0k 1.2M 1.2M 0.0k nvme5n20 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 117.6k 111.2k 0.0k 1.1M 1.1M 0.0k nvme5n3 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 110.4k 131.2k 0.0k 1.1M 1.3M 0.0k nvme5n4 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 140.8k 117.6k 0.0k 1.4M 1.1M 0.0k nvme5n5 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 144.0k 118.4k 0.0k 1.4M 1.2M 0.0k nvme5n6 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 117.2k 155.6k 0.0k 1.1M 1.5M 0.0k nvme5n7 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 114.8k 113.2k 0.0k 1.1M 1.1M 0.0k nvme5n8 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 132.4k 130.4k 0.0k 1.3M 1.3M 0.0k nvme5n9 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 126.8k 136.0k 0.0k 1.2M 1.3M 0.0k nvme8n1 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 126.0k 140.4k 0.0k 1.2M 1.4M 0.0k nvme8n10 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 154.4k 130.4k 0.0k 1.5M 1.3M 0.0k nvme8n11 2024-05-21T12:32:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 110.0k 151.2k 0.0k 1.1M 1.5M 0.0k nvme8n12 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 116.4k 141.6k 0.0k 1.1M 1.4M 0.0k nvme8n13 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 135.2k 140.0k 0.0k 1.3M 1.4M 0.0k nvme8n14 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 159.6k 115.6k 0.0k 1.6M 1.1M 0.0k nvme8n15 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 165.2k 108.4k 0.0k 1.6M 1.1M 0.0k nvme8n16 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 116.0k 131.2k 0.0k 1.1M 1.3M 0.0k nvme8n17 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 97.6k 145.2k 0.0k 976.0k 1.4M 0.0k nvme8n18 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 118.4k 146.4k 0.0k 1.2M 1.4M 0.0k nvme8n19 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 140.4k 111.6k 0.0k 1.4M 1.1M 0.0k nvme8n2 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 138.8k 114.0k 0.0k 1.4M 1.1M 0.0k nvme8n20 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 115.6k 150.4k 0.0k 1.1M 1.5M 0.0k nvme8n3 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 109.2k 132.8k 0.0k 1.1M 1.3M 0.0k nvme8n4 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 132.0k 116.0k 0.0k 1.3M 1.1M 0.0k nvme8n5 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 137.2k 123.6k 0.0k 1.3M 1.2M 0.0k nvme8n6 2024-05-21T12:32:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 98.0k 135.6k 0.0k 980.0k 1.3M 0.0k nvme8n7 2024-05-21T12:32:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 149.6k 117.2k 0.0k 1.5M 1.1M 0.0k nvme8n8 2024-05-21T12:32:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 129.2k 151.6k 0.0k 1.3M 1.5M 0.0k nvme8n9 2024-05-21T12:32:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:44.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:44 smithi072 ceph-mon[33912]: pgmap v505: 33 pgs: 33 active+clean; 1012 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-05-21T12:32:45.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:32:45] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 1 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: OPTIMIZED 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 1 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: OPTIMIZED 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:32:45.817 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 1 2024-05-21T12:32:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: OPTIMIZED 2024-05-21T12:32:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:32:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: ], , client address: IPv4 172.21.15.45:42396 2024-05-21T12:32:45.818 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:45.429380] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000252 seconds 2024-05-21T12:32:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:46 smithi092 ceph-mon[33665]: pgmap v506: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.4 MiB/s wr, 996 op/s 2024-05-21T12:32:46.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:46 smithi045 ceph-mon[30165]: pgmap v506: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.4 MiB/s wr, 996 op/s 2024-05-21T12:32:46.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:46 smithi072 ceph-mon[33912]: pgmap v506: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.4 MiB/s wr, 996 op/s 2024-05-21T12:32:48.717 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:48 smithi072 ceph-mon[33912]: pgmap v507: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 946 op/s 2024-05-21T12:32:48.717 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:48 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:32:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:48 smithi092 ceph-mon[33665]: pgmap v507: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 946 op/s 2024-05-21T12:32:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:48 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:32:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:48 smithi045 ceph-mon[30165]: pgmap v507: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 946 op/s 2024-05-21T12:32:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:48 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:32:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:32:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:32:49.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:32:49.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:32:49.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:32:49.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:32:49.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:32:49.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:32:49.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-05-21T12:32:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:32:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:32:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:32:49.916 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:32:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:50 smithi092 ceph-mon[33665]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-21T12:32:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:50 smithi092 ceph-mon[33665]: pgmap v508: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 892 op/s 2024-05-21T12:32:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:50 smithi045 ceph-mon[30165]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-21T12:32:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:50 smithi045 ceph-mon[30165]: pgmap v508: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 892 op/s 2024-05-21T12:32:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:50 smithi072 ceph-mon[33912]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-21T12:32:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:50 smithi072 ceph-mon[33912]: pgmap v508: 33 pgs: 33 active+clean; 1013 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 892 op/s 2024-05-21T12:32:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:52 smithi092 ceph-mon[33665]: pgmap v509: 33 pgs: 33 active+clean; 1014 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 806 op/s 2024-05-21T12:32:52.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:52 smithi045 ceph-mon[30165]: pgmap v509: 33 pgs: 33 active+clean; 1014 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 806 op/s 2024-05-21T12:32:52.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:52 smithi072 ceph-mon[33912]: pgmap v509: 33 pgs: 33 active+clean; 1014 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 806 op/s 2024-05-21T12:32:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:54 smithi092 ceph-mon[33665]: pgmap v510: 33 pgs: 33 active+clean; 1015 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 6.8 MiB/s wr, 950 op/s 2024-05-21T12:32:54.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:54 smithi045 ceph-mon[30165]: pgmap v510: 33 pgs: 33 active+clean; 1015 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 6.8 MiB/s wr, 950 op/s 2024-05-21T12:32:54.865 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:54.872 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 230.8k 110.0k 0.0k 2.3M 1.1M 0.0k nvme2n1 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 217.6k 130.0k 0.0k 2.1M 1.3M 0.0k nvme2n10 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 220.8k 130.4k 0.0k 2.2M 1.3M 0.0k nvme2n11 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 226.8k 137.6k 0.0k 2.2M 1.3M 0.0k nvme2n12 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 219.2k 130.4k 0.0k 2.1M 1.3M 0.0k nvme2n13 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 212.8k 140.8k 0.0k 2.1M 1.4M 0.0k nvme2n14 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 226.0k 113.2k 0.0k 2.2M 1.1M 0.0k nvme2n15 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 241.6k 116.8k 0.0k 2.4M 1.1M 0.0k nvme2n16 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 217.6k 120.4k 0.0k 2.1M 1.2M 0.0k nvme2n17 2024-05-21T12:32:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 244.4k 113.2k 0.0k 2.4M 1.1M 0.0k nvme2n18 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 258.0k 91.2k 0.0k 2.5M 912.0k 0.0k nvme2n19 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 231.6k 112.4k 0.0k 2.3M 1.1M 0.0k nvme2n2 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 218.0k 115.6k 0.0k 2.1M 1.1M 0.0k nvme2n20 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 206.4k 128.4k 0.0k 2.0M 1.3M 0.0k nvme2n3 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 216.4k 134.8k 0.0k 2.1M 1.3M 0.0k nvme2n4 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 231.2k 142.8k 0.0k 2.3M 1.4M 0.0k nvme2n5 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 228.0k 120.8k 0.0k 2.2M 1.2M 0.0k nvme2n6 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 232.4k 86.8k 0.0k 2.3M 868.0k 0.0k nvme2n7 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 230.8k 122.4k 0.0k 2.3M 1.2M 0.0k nvme2n8 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 208.8k 120.8k 0.0k 2.0M 1.2M 0.0k nvme2n9 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 234.8k 111.2k 0.0k 2.3M 1.1M 0.0k nvme5n1 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 200.8k 126.4k 0.0k 2.0M 1.2M 0.0k nvme5n10 2024-05-21T12:32:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 252.0k 113.2k 0.0k 2.5M 1.1M 0.0k nvme5n11 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 212.4k 122.8k 0.0k 2.1M 1.2M 0.0k nvme5n12 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 214.0k 130.0k 0.0k 2.1M 1.3M 0.0k nvme5n13 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 213.2k 122.8k 0.0k 2.1M 1.2M 0.0k nvme5n14 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 210.8k 106.8k 0.0k 2.1M 1.0M 0.0k nvme5n15 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 207.6k 116.4k 0.0k 2.0M 1.1M 0.0k nvme5n16 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 225.6k 118.0k 0.0k 2.2M 1.2M 0.0k nvme5n17 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 213.6k 130.0k 0.0k 2.1M 1.3M 0.0k nvme5n18 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 240.4k 100.4k 0.0k 2.3M 1004.0k 0.0k nvme5n19 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 208.4k 127.6k 0.0k 2.0M 1.2M 0.0k nvme5n2 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 218.8k 120.0k 0.0k 2.1M 1.2M 0.0k nvme5n20 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 198.4k 118.8k 0.0k 1.9M 1.2M 0.0k nvme5n3 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 218.8k 149.6k 0.0k 2.1M 1.5M 0.0k nvme5n4 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 211.2k 142.8k 0.0k 2.1M 1.4M 0.0k nvme5n5 2024-05-21T12:32:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 237.2k 111.2k 0.0k 2.3M 1.1M 0.0k nvme5n6 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 237.6k 88.0k 0.0k 2.3M 880.0k 0.0k nvme5n7 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 205.2k 118.4k 0.0k 2.0M 1.2M 0.0k nvme5n8 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 222.4k 115.6k 0.0k 2.2M 1.1M 0.0k nvme5n9 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 190.4k 167.6k 0.0k 1.9M 1.6M 0.0k nvme8n1 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 244.4k 140.0k 0.0k 2.4M 1.4M 0.0k nvme8n10 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 220.4k 115.6k 0.0k 2.2M 1.1M 0.0k nvme8n11 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 209.2k 130.0k 0.0k 2.0M 1.3M 0.0k nvme8n12 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 243.2k 140.0k 0.0k 2.4M 1.4M 0.0k nvme8n13 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.00 213.6k 107.6k 0.0k 2.1M 1.1M 0.0k nvme8n14 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 208.8k 145.2k 0.0k 2.0M 1.4M 0.0k nvme8n15 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 234.0k 109.6k 0.0k 2.3M 1.1M 0.0k nvme8n16 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 215.2k 115.6k 0.0k 2.1M 1.1M 0.0k nvme8n17 2024-05-21T12:32:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 226.0k 110.0k 0.0k 2.2M 1.1M 0.0k nvme8n18 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 220.0k 132.4k 0.0k 2.1M 1.3M 0.0k nvme8n19 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 213.2k 136.8k 0.0k 2.1M 1.3M 0.0k nvme8n2 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 249.6k 105.6k 0.0k 2.4M 1.0M 0.0k nvme8n20 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 232.8k 108.4k 0.0k 2.3M 1.1M 0.0k nvme8n3 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 227.6k 124.8k 0.0k 2.2M 1.2M 0.0k nvme8n4 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 216.4k 122.0k 0.0k 2.1M 1.2M 0.0k nvme8n5 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 243.6k 99.6k 0.0k 2.4M 996.0k 0.0k nvme8n6 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 208.4k 141.6k 0.0k 2.0M 1.4M 0.0k nvme8n7 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 208.8k 119.2k 0.0k 2.0M 1.2M 0.0k nvme8n8 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 17.10 219.6k 98.8k 0.0k 2.1M 988.0k 0.0k nvme8n9 2024-05-21T12:32:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:32:54.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:54 smithi072 ceph-mon[33912]: pgmap v510: 33 pgs: 33 active+clean; 1015 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 8.1 MiB/s rd, 6.8 MiB/s wr, 950 op/s 2024-05-21T12:32:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:56 smithi092 ceph-mon[33665]: pgmap v511: 33 pgs: 33 active+clean; 1016 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 770 op/s 2024-05-21T12:32:56.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:56 smithi045 ceph-mon[30165]: pgmap v511: 33 pgs: 33 active+clean; 1016 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 770 op/s 2024-05-21T12:32:56.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:56 smithi072 ceph-mon[33912]: pgmap v511: 33 pgs: 33 active+clean; 1016 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 770 op/s 2024-05-21T12:32:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:32:58 smithi092 ceph-mon[33665]: pgmap v512: 33 pgs: 33 active+clean; 1017 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 7.4 MiB/s rd, 6.4 MiB/s wr, 799 op/s 2024-05-21T12:32:58.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:32:58 smithi045 ceph-mon[30165]: pgmap v512: 33 pgs: 33 active+clean; 1017 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 7.4 MiB/s rd, 6.4 MiB/s wr, 799 op/s 2024-05-21T12:32:58.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:32:58 smithi072 ceph-mon[33912]: pgmap v512: 33 pgs: 33 active+clean; 1017 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 7.4 MiB/s rd, 6.4 MiB/s wr, 799 op/s 2024-05-21T12:33:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:00 smithi092 ceph-mon[33665]: pgmap v513: 33 pgs: 33 active+clean; 1018 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 856 op/s 2024-05-21T12:33:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:00 smithi045 ceph-mon[30165]: pgmap v513: 33 pgs: 33 active+clean; 1018 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 856 op/s 2024-05-21T12:33:00.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:00 smithi072 ceph-mon[33912]: pgmap v513: 33 pgs: 33 active+clean; 1018 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 856 op/s 2024-05-21T12:33:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:02 smithi092 ceph-mon[33665]: pgmap v514: 33 pgs: 33 active+clean; 1018 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 816 op/s 2024-05-21T12:33:02.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:02 smithi045 ceph-mon[30165]: pgmap v514: 33 pgs: 33 active+clean; 1018 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 816 op/s 2024-05-21T12:33:02.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:02 smithi072 ceph-mon[33912]: pgmap v514: 33 pgs: 33 active+clean; 1018 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 816 op/s 2024-05-21T12:33:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:04 smithi092 ceph-mon[33665]: pgmap v515: 33 pgs: 33 active+clean; 1019 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 9.0 MiB/s rd, 9.2 MiB/s wr, 960 op/s 2024-05-21T12:33:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:04 smithi045 ceph-mon[30165]: pgmap v515: 33 pgs: 33 active+clean; 1019 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 9.0 MiB/s rd, 9.2 MiB/s wr, 960 op/s 2024-05-21T12:33:04.865 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 183.6k 192.8k 0.0k 1.8M 1.9M 0.0k nvme2n1 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 168.4k 220.8k 0.0k 1.6M 2.2M 0.0k nvme2n10 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 195.2k 171.6k 0.0k 1.9M 1.7M 0.0k nvme2n11 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 192.8k 156.8k 0.0k 1.9M 1.5M 0.0k nvme2n12 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 172.8k 136.4k 0.0k 1.7M 1.3M 0.0k nvme2n13 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 174.8k 125.6k 0.0k 1.7M 1.2M 0.0k nvme2n14 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 178.8k 150.0k 0.0k 1.7M 1.5M 0.0k nvme2n15 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 163.2k 148.8k 0.0k 1.6M 1.5M 0.0k nvme2n16 2024-05-21T12:33:04.873 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 148.4k 168.8k 0.0k 1.4M 1.6M 0.0k nvme2n17 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 191.2k 156.4k 0.0k 1.9M 1.5M 0.0k nvme2n18 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 177.2k 171.2k 0.0k 1.7M 1.7M 0.0k nvme2n19 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 172.4k 195.6k 0.0k 1.7M 1.9M 0.0k nvme2n2 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 178.0k 209.6k 0.0k 1.7M 2.0M 0.0k nvme2n20 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 196.0k 152.4k 0.0k 1.9M 1.5M 0.0k nvme2n3 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 171.2k 182.4k 0.0k 1.7M 1.8M 0.0k nvme2n4 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 159.2k 179.6k 0.0k 1.6M 1.8M 0.0k nvme2n5 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 224.0k 139.6k 0.0k 2.2M 1.4M 0.0k nvme2n6 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 158.8k 157.6k 0.0k 1.6M 1.5M 0.0k nvme2n7 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 134.8k 120.8k 0.0k 1.3M 1.2M 0.0k nvme2n8 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 176.4k 171.2k 0.0k 1.7M 1.7M 0.0k nvme2n9 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 189.2k 177.2k 0.0k 1.8M 1.7M 0.0k nvme5n1 2024-05-21T12:33:04.874 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 169.2k 114.8k 0.0k 1.7M 1.1M 0.0k nvme5n10 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 144.4k 223.6k 0.0k 1.4M 2.2M 0.0k nvme5n11 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 141.2k 156.4k 0.0k 1.4M 1.5M 0.0k nvme5n12 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 198.4k 192.4k 0.0k 1.9M 1.9M 0.0k nvme5n13 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 183.2k 142.4k 0.0k 1.8M 1.4M 0.0k nvme5n14 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 166.4k 174.0k 0.0k 1.6M 1.7M 0.0k nvme5n15 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 214.0k 181.6k 0.0k 2.1M 1.8M 0.0k nvme5n16 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 202.4k 172.4k 0.0k 2.0M 1.7M 0.0k nvme5n17 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 134.0k 125.2k 0.0k 1.3M 1.2M 0.0k nvme5n18 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 184.0k 200.0k 0.0k 1.8M 2.0M 0.0k nvme5n19 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 145.6k 148.0k 0.0k 1.4M 1.4M 0.0k nvme5n2 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 132.8k 161.6k 0.0k 1.3M 1.6M 0.0k nvme5n20 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 139.6k 145.6k 0.0k 1.4M 1.4M 0.0k nvme5n3 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 151.6k 134.0k 0.0k 1.5M 1.3M 0.0k nvme5n4 2024-05-21T12:33:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 196.0k 147.2k 0.0k 1.9M 1.4M 0.0k nvme5n5 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 168.8k 186.8k 0.0k 1.6M 1.8M 0.0k nvme5n6 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 191.6k 167.6k 0.0k 1.9M 1.6M 0.0k nvme5n7 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 160.0k 124.4k 0.0k 1.6M 1.2M 0.0k nvme5n8 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 182.4k 184.4k 0.0k 1.8M 1.8M 0.0k nvme5n9 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 174.4k 160.0k 0.0k 1.7M 1.6M 0.0k nvme8n1 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 182.8k 208.4k 0.0k 1.8M 2.0M 0.0k nvme8n10 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 211.6k 234.8k 0.0k 2.1M 2.3M 0.0k nvme8n11 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 178.0k 140.4k 0.0k 1.7M 1.4M 0.0k nvme8n12 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 182.0k 148.8k 0.0k 1.8M 1.5M 0.0k nvme8n13 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.30 201.2k 200.4k 0.0k 2.0M 2.0M 0.0k nvme8n14 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 154.4k 191.2k 0.0k 1.5M 1.9M 0.0k nvme8n15 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 181.2k 194.0k 0.0k 1.8M 1.9M 0.0k nvme8n16 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 185.6k 151.6k 0.0k 1.8M 1.5M 0.0k nvme8n17 2024-05-21T12:33:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 205.6k 158.4k 0.0k 2.0M 1.5M 0.0k nvme8n18 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 130.4k 214.4k 0.0k 1.3M 2.1M 0.0k nvme8n19 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 255.2k 127.2k 0.0k 2.5M 1.2M 0.0k nvme8n2 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 182.8k 158.4k 0.0k 1.8M 1.5M 0.0k nvme8n20 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 186.4k 163.6k 0.0k 1.8M 1.6M 0.0k nvme8n3 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 155.6k 166.0k 0.0k 1.5M 1.6M 0.0k nvme8n4 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 163.2k 160.8k 0.0k 1.6M 1.6M 0.0k nvme8n5 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 164.4k 182.0k 0.0k 1.6M 1.8M 0.0k nvme8n6 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 192.8k 158.4k 0.0k 1.9M 1.5M 0.0k nvme8n7 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 215.2k 174.4k 0.0k 2.1M 1.7M 0.0k nvme8n8 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 14.20 149.2k 142.8k 0.0k 1.5M 1.4M 0.0k nvme8n9 2024-05-21T12:33:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:04.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:04 smithi072 ceph-mon[33912]: pgmap v515: 33 pgs: 33 active+clean; 1019 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 9.0 MiB/s rd, 9.2 MiB/s wr, 960 op/s 2024-05-21T12:33:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:06 smithi092 ceph-mon[33665]: pgmap v516: 33 pgs: 33 active+clean; 1019 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 8.7 MiB/s rd, 8.8 MiB/s wr, 866 op/s 2024-05-21T12:33:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:06 smithi045 ceph-mon[30165]: pgmap v516: 33 pgs: 33 active+clean; 1019 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 8.7 MiB/s rd, 8.8 MiB/s wr, 866 op/s 2024-05-21T12:33:06.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:06 smithi072 ceph-mon[33912]: pgmap v516: 33 pgs: 33 active+clean; 1019 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 8.7 MiB/s rd, 8.8 MiB/s wr, 866 op/s 2024-05-21T12:33:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:08 smithi092 ceph-mon[33665]: pgmap v517: 33 pgs: 33 active+clean; 1020 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 9.6 MiB/s rd, 9.7 MiB/s wr, 879 op/s 2024-05-21T12:33:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:08 smithi045 ceph-mon[30165]: pgmap v517: 33 pgs: 33 active+clean; 1020 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 9.6 MiB/s rd, 9.7 MiB/s wr, 879 op/s 2024-05-21T12:33:08.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:08 smithi072 ceph-mon[33912]: pgmap v517: 33 pgs: 33 active+clean; 1020 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 9.6 MiB/s rd, 9.7 MiB/s wr, 879 op/s 2024-05-21T12:33:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:10 smithi092 ceph-mon[33665]: pgmap v518: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 848 op/s 2024-05-21T12:33:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:10 smithi045 ceph-mon[30165]: pgmap v518: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 848 op/s 2024-05-21T12:33:10.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:10 smithi072 ceph-mon[33912]: pgmap v518: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 848 op/s 2024-05-21T12:33:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:12 smithi092 ceph-mon[33665]: pgmap v519: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 9.6 MiB/s rd, 9.9 MiB/s wr, 773 op/s 2024-05-21T12:33:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:12 smithi045 ceph-mon[30165]: pgmap v519: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 9.6 MiB/s rd, 9.9 MiB/s wr, 773 op/s 2024-05-21T12:33:12.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:12 smithi072 ceph-mon[33912]: pgmap v519: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 9.6 MiB/s rd, 9.9 MiB/s wr, 773 op/s 2024-05-21T12:33:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:14 smithi092 ceph-mon[33665]: pgmap v520: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 12 MiB/s wr, 891 op/s 2024-05-21T12:33:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:14 smithi045 ceph-mon[30165]: pgmap v520: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 12 MiB/s wr, 891 op/s 2024-05-21T12:33:14.865 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:33:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 166.8k 198.4k 0.0k 1.6M 1.9M 0.0k nvme2n1 2024-05-21T12:33:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 181.6k 176.4k 0.0k 1.8M 1.7M 0.0k nvme2n10 2024-05-21T12:33:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 172.4k 187.2k 0.0k 1.7M 1.8M 0.0k nvme2n11 2024-05-21T12:33:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 172.4k 222.4k 0.0k 1.7M 2.2M 0.0k nvme2n12 2024-05-21T12:33:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 206.0k 213.6k 0.0k 2.0M 2.1M 0.0k nvme2n13 2024-05-21T12:33:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 180.0k 167.6k 0.0k 1.8M 1.6M 0.0k nvme2n14 2024-05-21T12:33:14.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 204.0k 232.4k 0.0k 2.0M 2.3M 0.0k nvme2n15 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 220.0k 159.6k 0.0k 2.1M 1.6M 0.0k nvme2n16 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 187.2k 224.8k 0.0k 1.8M 2.2M 0.0k nvme2n17 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 182.0k 191.2k 0.0k 1.8M 1.9M 0.0k nvme2n18 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 172.0k 159.2k 0.0k 1.7M 1.6M 0.0k nvme2n19 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 162.0k 202.8k 0.0k 1.6M 2.0M 0.0k nvme2n2 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 172.8k 178.8k 0.0k 1.7M 1.7M 0.0k nvme2n20 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 194.4k 197.2k 0.0k 1.9M 1.9M 0.0k nvme2n3 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 155.2k 171.6k 0.0k 1.5M 1.7M 0.0k nvme2n4 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 159.6k 233.6k 0.0k 1.6M 2.3M 0.0k nvme2n5 2024-05-21T12:33:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 174.0k 200.8k 0.0k 1.7M 2.0M 0.0k nvme2n6 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 232.4k 182.0k 0.0k 2.3M 1.8M 0.0k nvme2n7 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 196.0k 252.0k 0.0k 1.9M 2.5M 0.0k nvme2n8 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 218.8k 156.0k 0.0k 2.1M 1.5M 0.0k nvme2n9 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 183.6k 193.6k 0.0k 1.8M 1.9M 0.0k nvme5n1 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 197.2k 182.8k 0.0k 1.9M 1.8M 0.0k nvme5n10 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 157.6k 215.6k 0.0k 1.5M 2.1M 0.0k nvme5n11 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 191.2k 183.6k 0.0k 1.9M 1.8M 0.0k nvme5n12 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 152.8k 221.6k 0.0k 1.5M 2.2M 0.0k nvme5n13 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 172.8k 172.8k 0.0k 1.7M 1.7M 0.0k nvme5n14 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 212.0k 156.0k 0.0k 2.1M 1.5M 0.0k nvme5n15 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 182.0k 171.6k 0.0k 1.8M 1.7M 0.0k nvme5n16 2024-05-21T12:33:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 164.4k 245.2k 0.0k 1.6M 2.4M 0.0k nvme5n17 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 216.4k 208.0k 0.0k 2.1M 2.0M 0.0k nvme5n18 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 176.0k 178.0k 0.0k 1.7M 1.7M 0.0k nvme5n19 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 180.0k 230.4k 0.0k 1.8M 2.2M 0.0k nvme5n2 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 190.8k 179.6k 0.0k 1.9M 1.8M 0.0k nvme5n20 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 224.0k 188.4k 0.0k 2.2M 1.8M 0.0k nvme5n3 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 175.6k 208.0k 0.0k 1.7M 2.0M 0.0k nvme5n4 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 183.2k 178.8k 0.0k 1.8M 1.7M 0.0k nvme5n5 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 214.0k 168.8k 0.0k 2.1M 1.6M 0.0k nvme5n6 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 191.6k 169.2k 0.0k 1.9M 1.7M 0.0k nvme5n7 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 171.6k 213.2k 0.0k 1.7M 2.1M 0.0k nvme5n8 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 200.4k 172.4k 0.0k 2.0M 1.7M 0.0k nvme5n9 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 177.6k 217.6k 0.0k 1.7M 2.1M 0.0k nvme8n1 2024-05-21T12:33:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 224.8k 146.8k 0.0k 2.2M 1.4M 0.0k nvme8n10 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 168.4k 171.6k 0.0k 1.6M 1.7M 0.0k nvme8n11 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 143.2k 230.8k 0.0k 1.4M 2.3M 0.0k nvme8n12 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 163.2k 193.2k 0.0k 1.6M 1.9M 0.0k nvme8n13 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 217.2k 156.8k 0.0k 2.1M 1.5M 0.0k nvme8n14 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 149.6k 193.6k 0.0k 1.5M 1.9M 0.0k nvme8n15 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 168.8k 188.8k 0.0k 1.6M 1.8M 0.0k nvme8n16 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 196.0k 196.4k 0.0k 1.9M 1.9M 0.0k nvme8n17 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 166.4k 200.8k 0.0k 1.6M 2.0M 0.0k nvme8n18 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 167.2k 184.0k 0.0k 1.6M 1.8M 0.0k nvme8n19 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 187.2k 191.6k 0.0k 1.8M 1.9M 0.0k nvme8n2 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 191.6k 208.0k 0.0k 1.9M 2.0M 0.0k nvme8n20 2024-05-21T12:33:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 199.6k 190.8k 0.0k 1.9M 1.9M 0.0k nvme8n3 2024-05-21T12:33:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 184.0k 180.8k 0.0k 1.8M 1.8M 0.0k nvme8n4 2024-05-21T12:33:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 173.2k 204.4k 0.0k 1.7M 2.0M 0.0k nvme8n5 2024-05-21T12:33:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 187.6k 179.2k 0.0k 1.8M 1.8M 0.0k nvme8n6 2024-05-21T12:33:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 154.8k 186.0k 0.0k 1.5M 1.8M 0.0k nvme8n7 2024-05-21T12:33:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 208.0k 188.4k 0.0k 2.0M 1.8M 0.0k nvme8n8 2024-05-21T12:33:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 232.0k 172.8k 0.0k 2.3M 1.7M 0.0k nvme8n9 2024-05-21T12:33:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:14.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:14 smithi072 ceph-mon[33912]: pgmap v520: 33 pgs: 33 active+clean; 1020 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 12 MiB/s wr, 891 op/s 2024-05-21T12:33:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:16 smithi092 ceph-mon[33665]: pgmap v521: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-05-21T12:33:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:16 smithi045 ceph-mon[30165]: pgmap v521: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-05-21T12:33:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:16 smithi072 ceph-mon[33912]: pgmap v521: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-05-21T12:33:18.717 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:18 smithi072 ceph-mon[33912]: pgmap v522: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 809 op/s 2024-05-21T12:33:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:18 smithi092 ceph-mon[33665]: pgmap v522: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 809 op/s 2024-05-21T12:33:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:18 smithi045 ceph-mon[30165]: pgmap v522: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 809 op/s 2024-05-21T12:33:20.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:20 smithi092 ceph-mon[33665]: pgmap v523: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 794 op/s 2024-05-21T12:33:20.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:20 smithi045 ceph-mon[30165]: pgmap v523: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 794 op/s 2024-05-21T12:33:20.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:20 smithi072 ceph-mon[33912]: pgmap v523: 33 pgs: 33 active+clean; 1020 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 11 MiB/s wr, 794 op/s 2024-05-21T12:33:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:22 smithi092 ceph-mon[33665]: pgmap v524: 33 pgs: 33 active+clean; 1021 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 9.2 MiB/s rd, 9.9 MiB/s wr, 721 op/s 2024-05-21T12:33:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:22 smithi045 ceph-mon[30165]: pgmap v524: 33 pgs: 33 active+clean; 1021 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 9.2 MiB/s rd, 9.9 MiB/s wr, 721 op/s 2024-05-21T12:33:22.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:22 smithi072 ceph-mon[33912]: pgmap v524: 33 pgs: 33 active+clean; 1021 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 9.2 MiB/s rd, 9.9 MiB/s wr, 721 op/s 2024-05-21T12:33:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:24 smithi092 ceph-mon[33665]: pgmap v525: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-05-21T12:33:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:24 smithi045 ceph-mon[30165]: pgmap v525: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-05-21T12:33:24.865 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:24.872 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 179.6k 184.0k 0.0k 1.8M 1.8M 0.0k nvme2n1 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 166.8k 141.2k 0.0k 1.6M 1.4M 0.0k nvme2n10 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 171.2k 197.2k 0.0k 1.7M 1.9M 0.0k nvme2n11 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 148.8k 182.4k 0.0k 1.5M 1.8M 0.0k nvme2n12 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 190.8k 149.6k 0.0k 1.9M 1.5M 0.0k nvme2n13 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 198.4k 149.2k 0.0k 1.9M 1.5M 0.0k nvme2n14 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 159.6k 152.4k 0.0k 1.6M 1.5M 0.0k nvme2n15 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 167.2k 176.4k 0.0k 1.6M 1.7M 0.0k nvme2n16 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 191.2k 120.0k 0.0k 1.9M 1.2M 0.0k nvme2n17 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 158.8k 182.4k 0.0k 1.6M 1.8M 0.0k nvme2n18 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 153.6k 181.2k 0.0k 1.5M 1.8M 0.0k nvme2n19 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 192.4k 162.0k 0.0k 1.9M 1.6M 0.0k nvme2n2 2024-05-21T12:33:24.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 184.0k 187.6k 0.0k 1.8M 1.8M 0.0k nvme2n20 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 176.0k 159.6k 0.0k 1.7M 1.6M 0.0k nvme2n3 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 144.0k 216.4k 0.0k 1.4M 2.1M 0.0k nvme2n4 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 174.4k 170.8k 0.0k 1.7M 1.7M 0.0k nvme2n5 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 179.2k 163.6k 0.0k 1.8M 1.6M 0.0k nvme2n6 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 189.2k 143.2k 0.0k 1.8M 1.4M 0.0k nvme2n7 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 149.2k 188.8k 0.0k 1.5M 1.8M 0.0k nvme2n8 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 174.8k 168.4k 0.0k 1.7M 1.6M 0.0k nvme2n9 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 181.6k 174.8k 0.0k 1.8M 1.7M 0.0k nvme5n1 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 162.8k 176.4k 0.0k 1.6M 1.7M 0.0k nvme5n10 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 157.2k 199.2k 0.0k 1.5M 1.9M 0.0k nvme5n11 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 196.0k 165.6k 0.0k 1.9M 1.6M 0.0k nvme5n12 2024-05-21T12:33:24.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 202.8k 150.0k 0.0k 2.0M 1.5M 0.0k nvme5n13 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 157.2k 184.0k 0.0k 1.5M 1.8M 0.0k nvme5n14 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 187.2k 157.2k 0.0k 1.8M 1.5M 0.0k nvme5n15 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 163.6k 192.8k 0.0k 1.6M 1.9M 0.0k nvme5n16 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 174.0k 150.8k 0.0k 1.7M 1.5M 0.0k nvme5n17 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 152.0k 197.6k 0.0k 1.5M 1.9M 0.0k nvme5n18 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 134.4k 149.2k 0.0k 1.3M 1.5M 0.0k nvme5n19 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 182.8k 192.4k 0.0k 1.8M 1.9M 0.0k nvme5n2 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 171.2k 170.0k 0.0k 1.7M 1.7M 0.0k nvme5n20 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 166.8k 181.2k 0.0k 1.6M 1.8M 0.0k nvme5n3 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 197.2k 161.6k 0.0k 1.9M 1.6M 0.0k nvme5n4 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 184.4k 162.4k 0.0k 1.8M 1.6M 0.0k nvme5n5 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 163.6k 138.8k 0.0k 1.6M 1.4M 0.0k nvme5n6 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 200.4k 169.6k 0.0k 2.0M 1.7M 0.0k nvme5n7 2024-05-21T12:33:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 146.0k 206.0k 0.0k 1.4M 2.0M 0.0k nvme5n8 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 151.6k 193.2k 0.0k 1.5M 1.9M 0.0k nvme5n9 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 135.6k 200.0k 0.0k 1.3M 2.0M 0.0k nvme8n1 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 179.6k 158.4k 0.0k 1.8M 1.5M 0.0k nvme8n10 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 132.0k 194.8k 0.0k 1.3M 1.9M 0.0k nvme8n11 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 181.6k 162.8k 0.0k 1.8M 1.6M 0.0k nvme8n12 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 140.4k 171.2k 0.0k 1.4M 1.7M 0.0k nvme8n13 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 218.0k 152.8k 0.0k 2.1M 1.5M 0.0k nvme8n14 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 158.8k 166.4k 0.0k 1.6M 1.6M 0.0k nvme8n15 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 143.6k 189.6k 0.0k 1.4M 1.9M 0.0k nvme8n16 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.10 184.4k 156.8k 0.0k 1.8M 1.5M 0.0k nvme8n17 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 156.8k 184.0k 0.0k 1.5M 1.8M 0.0k nvme8n18 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 170.0k 152.8k 0.0k 1.7M 1.5M 0.0k nvme8n19 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 162.0k 153.2k 0.0k 1.6M 1.5M 0.0k nvme8n2 2024-05-21T12:33:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 178.8k 160.4k 0.0k 1.7M 1.6M 0.0k nvme8n20 2024-05-21T12:33:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 160.0k 153.6k 0.0k 1.6M 1.5M 0.0k nvme8n3 2024-05-21T12:33:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 161.6k 192.0k 0.0k 1.6M 1.9M 0.0k nvme8n4 2024-05-21T12:33:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 198.8k 169.2k 0.0k 1.9M 1.7M 0.0k nvme8n5 2024-05-21T12:33:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 156.8k 201.6k 0.0k 1.5M 2.0M 0.0k nvme8n6 2024-05-21T12:33:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 171.2k 167.2k 0.0k 1.7M 1.6M 0.0k nvme8n7 2024-05-21T12:33:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 206.4k 176.8k 0.0k 2.0M 1.7M 0.0k nvme8n8 2024-05-21T12:33:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 167.2k 156.8k 0.0k 1.6M 1.5M 0.0k nvme8n9 2024-05-21T12:33:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:24.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:24 smithi072 ceph-mon[33912]: pgmap v525: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-05-21T12:33:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:26 smithi092 ceph-mon[33665]: pgmap v526: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 780 op/s 2024-05-21T12:33:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:26 smithi045 ceph-mon[30165]: pgmap v526: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 780 op/s 2024-05-21T12:33:26.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:26 smithi072 ceph-mon[33912]: pgmap v526: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 780 op/s 2024-05-21T12:33:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:28 smithi092 ceph-mon[33665]: pgmap v527: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 798 op/s 2024-05-21T12:33:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:28 smithi045 ceph-mon[30165]: pgmap v527: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 798 op/s 2024-05-21T12:33:28.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:28 smithi072 ceph-mon[33912]: pgmap v527: 33 pgs: 33 active+clean; 1021 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 798 op/s 2024-05-21T12:33:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:30 smithi092 ceph-mon[33665]: pgmap v528: 33 pgs: 33 active+clean; 1021 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.1 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2024-05-21T12:33:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:30 smithi045 ceph-mon[30165]: pgmap v528: 33 pgs: 33 active+clean; 1021 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.1 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2024-05-21T12:33:30.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:30 smithi072 ceph-mon[33912]: pgmap v528: 33 pgs: 33 active+clean; 1021 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.1 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2024-05-21T12:33:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:32 smithi092 ceph-mon[33665]: pgmap v529: 33 pgs: 33 active+clean; 1022 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 717 op/s 2024-05-21T12:33:32.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:32 smithi045 ceph-mon[30165]: pgmap v529: 33 pgs: 33 active+clean; 1022 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 717 op/s 2024-05-21T12:33:32.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:32 smithi072 ceph-mon[33912]: pgmap v529: 33 pgs: 33 active+clean; 1022 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 717 op/s 2024-05-21T12:33:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:34 smithi092 ceph-mon[33665]: pgmap v530: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 850 op/s 2024-05-21T12:33:34.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:34 smithi045 ceph-mon[30165]: pgmap v530: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 850 op/s 2024-05-21T12:33:34.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:34.873 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:33:34.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 132.0k 180.8k 0.0k 1.3M 1.8M 0.0k nvme2n1 2024-05-21T12:33:34.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 161.2k 159.2k 0.0k 1.6M 1.6M 0.0k nvme2n10 2024-05-21T12:33:34.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 153.6k 119.2k 0.0k 1.5M 1.2M 0.0k nvme2n11 2024-05-21T12:33:34.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 155.2k 118.8k 0.0k 1.5M 1.2M 0.0k nvme2n12 2024-05-21T12:33:34.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 182.0k 143.2k 0.0k 1.8M 1.4M 0.0k nvme2n13 2024-05-21T12:33:34.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 166.8k 149.2k 0.0k 1.6M 1.5M 0.0k nvme2n14 2024-05-21T12:33:34.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 169.6k 137.2k 0.0k 1.7M 1.3M 0.0k nvme2n15 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 162.8k 133.6k 0.0k 1.6M 1.3M 0.0k nvme2n16 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 168.8k 144.0k 0.0k 1.6M 1.4M 0.0k nvme2n17 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 134.8k 131.6k 0.0k 1.3M 1.3M 0.0k nvme2n18 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 161.6k 125.6k 0.0k 1.6M 1.2M 0.0k nvme2n19 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 161.6k 111.2k 0.0k 1.6M 1.1M 0.0k nvme2n2 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 161.2k 126.8k 0.0k 1.6M 1.2M 0.0k nvme2n20 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 154.8k 142.8k 0.0k 1.5M 1.4M 0.0k nvme2n3 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 164.0k 154.0k 0.0k 1.6M 1.5M 0.0k nvme2n4 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 136.8k 167.6k 0.0k 1.3M 1.6M 0.0k nvme2n5 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 172.0k 134.0k 0.0k 1.7M 1.3M 0.0k nvme2n6 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 146.8k 154.0k 0.0k 1.4M 1.5M 0.0k nvme2n7 2024-05-21T12:33:34.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 165.2k 156.4k 0.0k 1.6M 1.5M 0.0k nvme2n8 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 142.0k 129.6k 0.0k 1.4M 1.3M 0.0k nvme2n9 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 119.2k 189.6k 0.0k 1.2M 1.9M 0.0k nvme5n1 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 141.6k 144.4k 0.0k 1.4M 1.4M 0.0k nvme5n10 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 146.4k 161.6k 0.0k 1.4M 1.6M 0.0k nvme5n11 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 132.0k 128.8k 0.0k 1.3M 1.3M 0.0k nvme5n12 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 129.6k 166.0k 0.0k 1.3M 1.6M 0.0k nvme5n13 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 194.0k 142.4k 0.0k 1.9M 1.4M 0.0k nvme5n14 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 152.4k 140.0k 0.0k 1.5M 1.4M 0.0k nvme5n15 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 120.8k 160.8k 0.0k 1.2M 1.6M 0.0k nvme5n16 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 181.6k 146.8k 0.0k 1.8M 1.4M 0.0k nvme5n17 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 155.6k 164.0k 0.0k 1.5M 1.6M 0.0k nvme5n18 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 138.8k 168.4k 0.0k 1.4M 1.6M 0.0k nvme5n19 2024-05-21T12:33:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 162.4k 142.8k 0.0k 1.6M 1.4M 0.0k nvme5n2 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 125.6k 166.8k 0.0k 1.2M 1.6M 0.0k nvme5n20 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 146.8k 169.6k 0.0k 1.4M 1.7M 0.0k nvme5n3 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 154.8k 144.0k 0.0k 1.5M 1.4M 0.0k nvme5n4 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 119.2k 172.4k 0.0k 1.2M 1.7M 0.0k nvme5n5 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 157.2k 134.4k 0.0k 1.5M 1.3M 0.0k nvme5n6 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 172.0k 157.6k 0.0k 1.7M 1.5M 0.0k nvme5n7 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 148.4k 180.4k 0.0k 1.4M 1.8M 0.0k nvme5n8 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 148.8k 175.2k 0.0k 1.5M 1.7M 0.0k nvme5n9 2024-05-21T12:33:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 137.2k 149.6k 0.0k 1.3M 1.5M 0.0k nvme8n1 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 154.4k 138.0k 0.0k 1.5M 1.3M 0.0k nvme8n10 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 136.0k 158.0k 0.0k 1.3M 1.5M 0.0k nvme8n11 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 156.4k 132.4k 0.0k 1.5M 1.3M 0.0k nvme8n12 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 147.6k 147.2k 0.0k 1.4M 1.4M 0.0k nvme8n13 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 140.4k 119.2k 0.0k 1.4M 1.2M 0.0k nvme8n14 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 168.8k 150.4k 0.0k 1.6M 1.5M 0.0k nvme8n15 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 121.2k 186.4k 0.0k 1.2M 1.8M 0.0k nvme8n16 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 144.8k 127.6k 0.0k 1.4M 1.2M 0.0k nvme8n17 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 154.8k 166.0k 0.0k 1.5M 1.6M 0.0k nvme8n18 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 103.6k 191.6k 0.0k 1.0M 1.9M 0.0k nvme8n19 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 158.4k 146.4k 0.0k 1.5M 1.4M 0.0k nvme8n2 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 163.6k 164.0k 0.0k 1.6M 1.6M 0.0k nvme8n20 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 154.4k 138.4k 0.0k 1.5M 1.4M 0.0k nvme8n3 2024-05-21T12:33:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 127.6k 159.6k 0.0k 1.2M 1.6M 0.0k nvme8n4 2024-05-21T12:33:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 167.6k 126.4k 0.0k 1.6M 1.2M 0.0k nvme8n5 2024-05-21T12:33:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 130.4k 177.6k 0.0k 1.3M 1.7M 0.0k nvme8n6 2024-05-21T12:33:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 124.4k 154.8k 0.0k 1.2M 1.5M 0.0k nvme8n7 2024-05-21T12:33:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 167.2k 116.4k 0.0k 1.6M 1.1M 0.0k nvme8n8 2024-05-21T12:33:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 146.4k 160.0k 0.0k 1.4M 1.6M 0.0k nvme8n9 2024-05-21T12:33:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:34.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:34 smithi072 ceph-mon[33912]: pgmap v530: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 850 op/s 2024-05-21T12:33:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:36 smithi092 ceph-mon[33665]: pgmap v531: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.7 MiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-05-21T12:33:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:36 smithi045 ceph-mon[30165]: pgmap v531: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.7 MiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-05-21T12:33:36.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:36 smithi072 ceph-mon[33912]: pgmap v531: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.7 MiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-05-21T12:33:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:38 smithi092 ceph-mon[33665]: pgmap v532: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.8 MiB/s rd, 9.0 MiB/s wr, 788 op/s 2024-05-21T12:33:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:38 smithi045 ceph-mon[30165]: pgmap v532: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.8 MiB/s rd, 9.0 MiB/s wr, 788 op/s 2024-05-21T12:33:38.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:38 smithi072 ceph-mon[33912]: pgmap v532: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.8 MiB/s rd, 9.0 MiB/s wr, 788 op/s 2024-05-21T12:33:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:40 smithi092 ceph-mon[33665]: pgmap v533: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 737 op/s 2024-05-21T12:33:40.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:40 smithi045 ceph-mon[30165]: pgmap v533: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 737 op/s 2024-05-21T12:33:40.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:40 smithi072 ceph-mon[33912]: pgmap v533: 33 pgs: 33 active+clean; 1022 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 737 op/s 2024-05-21T12:33:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:42 smithi092 ceph-mon[33665]: pgmap v534: 33 pgs: 33 active+clean; 1022 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 750 op/s 2024-05-21T12:33:42.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:42 smithi045 ceph-mon[30165]: pgmap v534: 33 pgs: 33 active+clean; 1022 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 750 op/s 2024-05-21T12:33:42.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:42 smithi072 ceph-mon[33912]: pgmap v534: 33 pgs: 33 active+clean; 1022 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 750 op/s 2024-05-21T12:33:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:44 smithi092 ceph-mon[33665]: pgmap v535: 33 pgs: 33 active+clean; 1022 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 840 op/s 2024-05-21T12:33:44.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:44 smithi045 ceph-mon[30165]: pgmap v535: 33 pgs: 33 active+clean; 1022 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 840 op/s 2024-05-21T12:33:44.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:33:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 121.2k 159.2k 0.0k 1.2M 1.6M 0.0k nvme2n1 2024-05-21T12:33:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 110.4k 160.4k 0.0k 1.1M 1.6M 0.0k nvme2n10 2024-05-21T12:33:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 116.8k 136.0k 0.0k 1.1M 1.3M 0.0k nvme2n11 2024-05-21T12:33:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 138.0k 109.2k 0.0k 1.3M 1.1M 0.0k nvme2n12 2024-05-21T12:33:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 124.4k 123.6k 0.0k 1.2M 1.2M 0.0k nvme2n13 2024-05-21T12:33:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 146.8k 135.2k 0.0k 1.4M 1.3M 0.0k nvme2n14 2024-05-21T12:33:44.873 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 128.0k 136.0k 0.0k 1.2M 1.3M 0.0k nvme2n15 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 158.4k 123.2k 0.0k 1.5M 1.2M 0.0k nvme2n16 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 132.8k 106.8k 0.0k 1.3M 1.0M 0.0k nvme2n17 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 140.8k 155.2k 0.0k 1.4M 1.5M 0.0k nvme2n18 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 114.4k 127.2k 0.0k 1.1M 1.2M 0.0k nvme2n19 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 154.8k 132.0k 0.0k 1.5M 1.3M 0.0k nvme2n2 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 153.6k 110.4k 0.0k 1.5M 1.1M 0.0k nvme2n20 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 137.2k 119.6k 0.0k 1.3M 1.2M 0.0k nvme2n3 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 131.6k 109.6k 0.0k 1.3M 1.1M 0.0k nvme2n4 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 155.6k 102.4k 0.0k 1.5M 1.0M 0.0k nvme2n5 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 130.8k 126.4k 0.0k 1.3M 1.2M 0.0k nvme2n6 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 156.4k 130.8k 0.0k 1.5M 1.3M 0.0k nvme2n7 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 146.8k 170.8k 0.0k 1.4M 1.7M 0.0k nvme2n8 2024-05-21T12:33:44.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 134.4k 127.6k 0.0k 1.3M 1.2M 0.0k nvme2n9 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 107.2k 149.6k 0.0k 1.0M 1.5M 0.0k nvme5n1 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 169.6k 127.2k 0.0k 1.7M 1.2M 0.0k nvme5n10 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 128.8k 143.2k 0.0k 1.3M 1.4M 0.0k nvme5n11 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 153.2k 146.4k 0.0k 1.5M 1.4M 0.0k nvme5n12 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 104.8k 124.8k 0.0k 1.0M 1.2M 0.0k nvme5n13 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 120.8k 120.0k 0.0k 1.2M 1.2M 0.0k nvme5n14 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 150.0k 121.6k 0.0k 1.5M 1.2M 0.0k nvme5n15 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 132.0k 124.0k 0.0k 1.3M 1.2M 0.0k nvme5n16 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 119.2k 135.6k 0.0k 1.2M 1.3M 0.0k nvme5n17 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 138.8k 123.2k 0.0k 1.4M 1.2M 0.0k nvme5n18 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 107.6k 126.4k 0.0k 1.1M 1.2M 0.0k nvme5n19 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 135.6k 142.0k 0.0k 1.3M 1.4M 0.0k nvme5n2 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 121.2k 172.8k 0.0k 1.2M 1.7M 0.0k nvme5n20 2024-05-21T12:33:44.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 156.0k 106.0k 0.0k 1.5M 1.0M 0.0k nvme5n3 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 140.0k 112.8k 0.0k 1.4M 1.1M 0.0k nvme5n4 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 144.4k 144.0k 0.0k 1.4M 1.4M 0.0k nvme5n5 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 108.8k 145.6k 0.0k 1.1M 1.4M 0.0k nvme5n6 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 128.0k 133.2k 0.0k 1.2M 1.3M 0.0k nvme5n7 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 146.4k 144.0k 0.0k 1.4M 1.4M 0.0k nvme5n8 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 130.8k 143.2k 0.0k 1.3M 1.4M 0.0k nvme5n9 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 137.6k 145.2k 0.0k 1.3M 1.4M 0.0k nvme8n1 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 148.0k 120.8k 0.0k 1.4M 1.2M 0.0k nvme8n10 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 138.8k 116.0k 0.0k 1.4M 1.1M 0.0k nvme8n11 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 162.8k 82.4k 0.0k 1.6M 824.0k 0.0k nvme8n12 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 123.2k 133.2k 0.0k 1.2M 1.3M 0.0k nvme8n13 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 126.0k 147.2k 0.0k 1.2M 1.4M 0.0k nvme8n14 2024-05-21T12:33:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 116.4k 157.2k 0.0k 1.1M 1.5M 0.0k nvme8n15 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 102.8k 131.2k 0.0k 1.0M 1.3M 0.0k nvme8n16 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 144.0k 98.8k 0.0k 1.4M 988.0k 0.0k nvme8n17 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 121.2k 138.8k 0.0k 1.2M 1.4M 0.0k nvme8n18 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 153.2k 143.2k 0.0k 1.5M 1.4M 0.0k nvme8n19 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 172.8k 93.6k 0.0k 1.7M 936.0k 0.0k nvme8n2 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 139.2k 126.8k 0.0k 1.4M 1.2M 0.0k nvme8n20 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 100.4k 128.0k 0.0k 1004.0k 1.2M 0.0k nvme8n3 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 146.8k 104.0k 0.0k 1.4M 1.0M 0.0k nvme8n4 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 119.2k 126.8k 0.0k 1.2M 1.2M 0.0k nvme8n5 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 132.0k 122.0k 0.0k 1.3M 1.2M 0.0k nvme8n6 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.70 124.8k 135.6k 0.0k 1.2M 1.3M 0.0k nvme8n7 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 110.8k 143.2k 0.0k 1.1M 1.4M 0.0k nvme8n8 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.80 139.6k 108.0k 0.0k 1.4M 1.1M 0.0k nvme8n9 2024-05-21T12:33:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:44.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:44 smithi072 ceph-mon[33912]: pgmap v535: 33 pgs: 33 active+clean; 1022 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 840 op/s 2024-05-21T12:33:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:46 smithi092 ceph-mon[33665]: pgmap v536: 33 pgs: 33 active+clean; 1023 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 763 op/s 2024-05-21T12:33:46.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:46 smithi045 ceph-mon[30165]: pgmap v536: 33 pgs: 33 active+clean; 1023 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 763 op/s 2024-05-21T12:33:46.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:46 smithi072 ceph-mon[33912]: pgmap v536: 33 pgs: 33 active+clean; 1023 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 763 op/s 2024-05-21T12:33:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:48 smithi092 ceph-mon[33665]: pgmap v537: 33 pgs: 33 active+clean; 1023 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 781 op/s 2024-05-21T12:33:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:48 smithi045 ceph-mon[30165]: pgmap v537: 33 pgs: 33 active+clean; 1023 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 781 op/s 2024-05-21T12:33:48.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:48 smithi072 ceph-mon[33912]: pgmap v537: 33 pgs: 33 active+clean; 1023 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 781 op/s 2024-05-21T12:33:49.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:49 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:33:49.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:49 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:33:49.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:49 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:33:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:50 smithi092 ceph-mon[33665]: pgmap v538: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 731 op/s 2024-05-21T12:33:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:33:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:33:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:50 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:33:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:50 smithi045 ceph-mon[30165]: pgmap v538: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 731 op/s 2024-05-21T12:33:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:33:50.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:33:50.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:50 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:33:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:50 smithi072 ceph-mon[33912]: pgmap v538: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 731 op/s 2024-05-21T12:33:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:33:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:33:50.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:50 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:33:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:52 smithi092 ceph-mon[33665]: pgmap v539: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-05-21T12:33:52.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:52 smithi045 ceph-mon[30165]: pgmap v539: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-05-21T12:33:52.919 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:52 smithi072 ceph-mon[33912]: pgmap v539: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-05-21T12:33:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:54 smithi092 ceph-mon[33665]: pgmap v540: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 837 op/s 2024-05-21T12:33:54.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:54 smithi045 ceph-mon[30165]: pgmap v540: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 837 op/s 2024-05-21T12:33:54.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:33:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 127.2k 132.4k 0.0k 1.2M 1.3M 0.0k nvme2n1 2024-05-21T12:33:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 109.2k 130.4k 0.0k 1.1M 1.3M 0.0k nvme2n10 2024-05-21T12:33:54.873 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 112.4k 98.4k 0.0k 1.1M 984.0k 0.0k nvme2n11 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 134.4k 124.8k 0.0k 1.3M 1.2M 0.0k nvme2n12 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 130.8k 117.2k 0.0k 1.3M 1.1M 0.0k nvme2n13 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 99.2k 116.0k 0.0k 992.0k 1.1M 0.0k nvme2n14 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 150.0k 125.6k 0.0k 1.5M 1.2M 0.0k nvme2n15 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 118.4k 128.8k 0.0k 1.2M 1.3M 0.0k nvme2n16 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 138.8k 110.4k 0.0k 1.4M 1.1M 0.0k nvme2n17 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 132.4k 122.0k 0.0k 1.3M 1.2M 0.0k nvme2n18 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 130.0k 143.6k 0.0k 1.3M 1.4M 0.0k nvme2n19 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 126.4k 135.2k 0.0k 1.2M 1.3M 0.0k nvme2n2 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 162.0k 150.8k 0.0k 1.6M 1.5M 0.0k nvme2n20 2024-05-21T12:33:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 136.0k 90.8k 0.0k 1.3M 908.0k 0.0k nvme2n3 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 134.0k 126.4k 0.0k 1.3M 1.2M 0.0k nvme2n4 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 132.4k 137.2k 0.0k 1.3M 1.3M 0.0k nvme2n5 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 120.0k 115.6k 0.0k 1.2M 1.1M 0.0k nvme2n6 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 150.0k 107.6k 0.0k 1.5M 1.1M 0.0k nvme2n7 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 118.4k 135.2k 0.0k 1.2M 1.3M 0.0k nvme2n8 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 117.2k 128.8k 0.0k 1.1M 1.3M 0.0k nvme2n9 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 139.2k 117.6k 0.0k 1.4M 1.1M 0.0k nvme5n1 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 123.2k 130.4k 0.0k 1.2M 1.3M 0.0k nvme5n10 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 131.2k 102.0k 0.0k 1.3M 1020.0k 0.0k nvme5n11 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 124.4k 126.4k 0.0k 1.2M 1.2M 0.0k nvme5n12 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 125.6k 126.0k 0.0k 1.2M 1.2M 0.0k nvme5n13 2024-05-21T12:33:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 137.2k 113.2k 0.0k 1.3M 1.1M 0.0k nvme5n14 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 107.2k 124.4k 0.0k 1.0M 1.2M 0.0k nvme5n15 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 120.4k 104.0k 0.0k 1.2M 1.0M 0.0k nvme5n16 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 118.8k 139.6k 0.0k 1.2M 1.4M 0.0k nvme5n17 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 128.0k 109.6k 0.0k 1.2M 1.1M 0.0k nvme5n18 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 104.4k 129.2k 0.0k 1.0M 1.3M 0.0k nvme5n19 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 136.4k 121.2k 0.0k 1.3M 1.2M 0.0k nvme5n2 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 119.2k 136.0k 0.0k 1.2M 1.3M 0.0k nvme5n20 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 116.8k 115.6k 0.0k 1.1M 1.1M 0.0k nvme5n3 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 131.6k 117.2k 0.0k 1.3M 1.1M 0.0k nvme5n4 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 131.2k 128.8k 0.0k 1.3M 1.3M 0.0k nvme5n5 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 112.0k 137.2k 0.0k 1.1M 1.3M 0.0k nvme5n6 2024-05-21T12:33:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 156.8k 91.6k 0.0k 1.5M 916.0k 0.0k nvme5n7 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 120.8k 126.0k 0.0k 1.2M 1.2M 0.0k nvme5n8 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 149.2k 142.0k 0.0k 1.5M 1.4M 0.0k nvme5n9 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 130.4k 126.4k 0.0k 1.3M 1.2M 0.0k nvme8n1 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 143.6k 136.8k 0.0k 1.4M 1.3M 0.0k nvme8n10 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 164.8k 102.8k 0.0k 1.6M 1.0M 0.0k nvme8n11 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 135.2k 114.8k 0.0k 1.3M 1.1M 0.0k nvme8n12 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 144.4k 107.2k 0.0k 1.4M 1.0M 0.0k nvme8n13 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 143.6k 137.2k 0.0k 1.4M 1.3M 0.0k nvme8n14 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.90 138.4k 104.0k 0.0k 1.4M 1.0M 0.0k nvme8n15 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 140.0k 136.8k 0.0k 1.4M 1.3M 0.0k nvme8n16 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 132.8k 113.2k 0.0k 1.3M 1.1M 0.0k nvme8n17 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 118.4k 130.4k 0.0k 1.2M 1.3M 0.0k nvme8n18 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 112.4k 136.4k 0.0k 1.1M 1.3M 0.0k nvme8n19 2024-05-21T12:33:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 120.4k 102.4k 0.0k 1.2M 1.0M 0.0k nvme8n2 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 121.2k 142.0k 0.0k 1.2M 1.4M 0.0k nvme8n20 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 116.4k 97.6k 0.0k 1.1M 976.0k 0.0k nvme8n3 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 124.4k 110.4k 0.0k 1.2M 1.1M 0.0k nvme8n4 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 128.8k 121.6k 0.0k 1.3M 1.2M 0.0k nvme8n5 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 121.6k 122.8k 0.0k 1.2M 1.2M 0.0k nvme8n6 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 112.8k 146.8k 0.0k 1.1M 1.4M 0.0k nvme8n7 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 164.0k 140.0k 0.0k 1.6M 1.4M 0.0k nvme8n8 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 13.00 132.4k 107.6k 0.0k 1.3M 1.1M 0.0k nvme8n9 2024-05-21T12:33:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:33:54.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:54 smithi072 ceph-mon[33912]: pgmap v540: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 837 op/s 2024-05-21T12:33:56.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:56 smithi092 ceph-mon[33665]: pgmap v541: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 758 op/s 2024-05-21T12:33:56.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:56 smithi045 ceph-mon[30165]: pgmap v541: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 758 op/s 2024-05-21T12:33:56.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:56 smithi072 ceph-mon[33912]: pgmap v541: 33 pgs: 33 active+clean; 1023 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 758 op/s 2024-05-21T12:33:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:33:58 smithi092 ceph-mon[33665]: pgmap v542: 33 pgs: 33 active+clean; 1023 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.3 MiB/s wr, 789 op/s 2024-05-21T12:33:58.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:33:58 smithi045 ceph-mon[30165]: pgmap v542: 33 pgs: 33 active+clean; 1023 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.3 MiB/s wr, 789 op/s 2024-05-21T12:33:58.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:33:58 smithi072 ceph-mon[33912]: pgmap v542: 33 pgs: 33 active+clean; 1023 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.3 MiB/s wr, 789 op/s 2024-05-21T12:34:00.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:00 smithi092 ceph-mon[33665]: pgmap v543: 33 pgs: 33 active+clean; 1023 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 6.9 MiB/s wr, 731 op/s 2024-05-21T12:34:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:00 smithi045 ceph-mon[30165]: pgmap v543: 33 pgs: 33 active+clean; 1023 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 6.9 MiB/s wr, 731 op/s 2024-05-21T12:34:00.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:00 smithi072 ceph-mon[33912]: pgmap v543: 33 pgs: 33 active+clean; 1023 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 6.9 MiB/s wr, 731 op/s 2024-05-21T12:34:02.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:02 smithi092 ceph-mon[33665]: pgmap v544: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.4 MiB/s wr, 745 op/s 2024-05-21T12:34:02.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:02 smithi045 ceph-mon[30165]: pgmap v544: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.4 MiB/s wr, 745 op/s 2024-05-21T12:34:02.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:02 smithi072 ceph-mon[33912]: pgmap v544: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.4 MiB/s wr, 745 op/s 2024-05-21T12:34:03.608 INFO:tasks.nvmeof.[nvmeof.thrasher]:done waiting before thrashing 2024-05-21T12:34:03.608 INFO:tasks.nvmeof.[nvmeof.thrasher]:display and verify stats: 2024-05-21T12:34:03.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:34:03.647 INFO:teuthology.orchestra.run.smithi045.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service - Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:34:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:34:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: Active: active (running) since Tue 2024-05-21 12:32:39 UTC; 1min 24s ago 2024-05-21T12:34:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 72823 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:34:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 72824 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: Main PID: 72885 (conmon) 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: Tasks: 126 (limit: 202854) 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: Memory: 139.9M 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: CPU: 1min 26.712s 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: ├─libpod-payload-c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─72887 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─72889 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─72906 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.a --gateway-address 172.21.15.45:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.45:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.a -k /etc/ceph/keyring 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─72954 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: │ └─73757 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: └─runtime 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: └─72885 /usr/bin/conmon --api-version 1 -c c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db -u c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db/userdata -p /run/containers/storage/overlay-containers/c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 1 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: OPTIMIZED 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: states { 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: grp_id: 1 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: state: OPTIMIZED 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: } 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: ], , client address: IPv4 172.21.15.45:42396 2024-05-21T12:34:03.649 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:32:45 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [2024-05-21 12:32:45.429380] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000252 seconds 2024-05-21T12:34:03.651 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:34:03.681 INFO:teuthology.orchestra.run.smithi072.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service - Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: Active: active (running) since Tue 2024-05-21 12:24:10 UTC; 9min ago 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 53097 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 53098 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: Main PID: 53263 (conmon) 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: Tasks: 126 (limit: 202814) 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: Memory: 1.0G 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: CPU: 9min 53.309s 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:34:03.682 INFO:teuthology.orchestra.run.smithi072.stdout: ├─libpod-payload-a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─53265 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─53267 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─53289 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.b --gateway-address 172.21.15.72:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.72:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.b -k /etc/ceph/keyring 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─53805 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout: │ └─53807 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout: └─runtime 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout: └─53263 /usr/bin/conmon --api-version 1 -c a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 -u a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243/userdata -p /run/containers/storage/overlay-containers/a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:34:03.683 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:34:03.684 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:34:03.684 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:34:03.684 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:34:03.685 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: states { 2024-05-21T12:34:03.685 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: grp_id: 1 2024-05-21T12:34:03.685 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: state: INACCESSIBLE 2024-05-21T12:34:03.685 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: } 2024-05-21T12:34:03.685 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:32:42 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: ], , client address: IPv4 172.21.15.72:59920 2024-05-21T12:34:03.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:34:03.717 INFO:teuthology.orchestra.run.smithi092.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service - Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:34:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: Active: active (running) since Tue 2024-05-21 12:23:56 UTC; 10min ago 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53502 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53503 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: Main PID: 53679 (conmon) 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: Tasks: 126 (limit: 202816) 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: Memory: 1.0G 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: CPU: 9min 51.675s 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: ├─libpod-payload-5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53681 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53683 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53701 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.c --gateway-address 172.21.15.92:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.92:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.c -k /etc/ceph/keyring 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─54233 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: │ └─54267 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: └─runtime 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: └─53679 /usr/bin/conmon --api-version 1 -c 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d -u 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata -p /run/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:53.769517] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:53] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:34:03.719 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:34:03.720 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:29:55.476877] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:34:03.720 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:29:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:29:55] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:34:03.720 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:30:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:30:00] INFO grpc.py:2407: Received request to list all subsystems, context: , client address: IPv4 172.21.15.137:52412 2024-05-21T12:34:03.721 DEBUG:teuthology.orchestra.run.smithi137:> ceph orch ls && ceph orch ps --daemon-type nvmeof && ceph health detail && ceph -s && sudo nvme list-subsys /dev/nvme5n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n20 | grep 'live optimized' 2024-05-21T12:34:04.036 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T12:34:04.036 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 1/1 9m ago 15m smithi045=x;count:1 2024-05-21T12:34:04.036 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 9m ago 15m smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:34:04.036 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.mypool ?:4420,5500,8009 3/3 9m ago 10m smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:34:04.036 INFO:teuthology.orchestra.run.smithi137.stdout:osd 8 9m ago - 2024-05-21T12:34:04.340 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:34:04.340 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.a smithi045 *:5500,4420,8009 running (9m) 9m ago 9m 47.9M - 1.2.9 30dd525ace30 4d34dcd5cab0 2024-05-21T12:34:04.340 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.b smithi072 *:5500,4420,8009 running (9m) 9m ago 9m 44.6M - 1.2.9 30dd525ace30 a048ad7abf63 2024-05-21T12:34:04.341 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.c smithi092 *:5500,4420,8009 running (10m) 9m ago 10m 45.1M - 1.2.9 30dd525ace30 5e5a8ffd2ba6 2024-05-21T12:34:04.701 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_OK 2024-05-21T12:34:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:04 smithi045 ceph-mon[30165]: pgmap v545: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.2 MiB/s rd, 9.1 MiB/s wr, 823 op/s 2024-05-21T12:34:04.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:34:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 154.4k 218.0k 0.0k 1.5M 2.1M 0.0k nvme2n1 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 220.4k 174.4k 0.0k 2.2M 1.7M 0.0k nvme2n10 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 172.8k 170.8k 0.0k 1.7M 1.7M 0.0k nvme2n11 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 175.2k 160.0k 0.0k 1.7M 1.6M 0.0k nvme2n12 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 180.8k 190.4k 0.0k 1.8M 1.9M 0.0k nvme2n13 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.00 153.6k 133.2k 0.0k 1.5M 1.3M 0.0k nvme2n14 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 12.00 152.4k 196.0k 0.0k 1.5M 1.9M 0.0k nvme2n15 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 151.2k 170.8k 0.0k 1.5M 1.7M 0.0k nvme2n16 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 158.4k 176.4k 0.0k 1.5M 1.7M 0.0k nvme2n17 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 167.6k 193.2k 0.0k 1.6M 1.9M 0.0k nvme2n18 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 164.4k 148.0k 0.0k 1.6M 1.4M 0.0k nvme2n19 2024-05-21T12:34:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 195.2k 149.6k 0.0k 1.9M 1.5M 0.0k nvme2n2 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 201.2k 189.2k 0.0k 2.0M 1.8M 0.0k nvme2n20 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 158.8k 163.6k 0.0k 1.6M 1.6M 0.0k nvme2n3 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 202.0k 155.6k 0.0k 2.0M 1.5M 0.0k nvme2n4 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 184.8k 159.2k 0.0k 1.8M 1.6M 0.0k nvme2n5 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 199.2k 187.6k 0.0k 1.9M 1.8M 0.0k nvme2n6 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 175.6k 200.4k 0.0k 1.7M 2.0M 0.0k nvme2n7 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 225.6k 184.8k 0.0k 2.2M 1.8M 0.0k nvme2n8 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 215.2k 159.6k 0.0k 2.1M 1.6M 0.0k nvme2n9 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 189.2k 173.6k 0.0k 1.8M 1.7M 0.0k nvme5n1 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 137.2k 181.6k 0.0k 1.3M 1.8M 0.0k nvme5n10 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 190.4k 190.0k 0.0k 1.9M 1.9M 0.0k nvme5n11 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 125.2k 171.6k 0.0k 1.2M 1.7M 0.0k nvme5n12 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 179.2k 189.6k 0.0k 1.8M 1.9M 0.0k nvme5n13 2024-05-21T12:34:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 12.00 132.4k 160.4k 0.0k 1.3M 1.6M 0.0k nvme5n14 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.00 150.0k 160.4k 0.0k 1.5M 1.6M 0.0k nvme5n15 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 12.00 194.4k 181.6k 0.0k 1.9M 1.8M 0.0k nvme5n16 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 196.0k 178.8k 0.0k 1.9M 1.7M 0.0k nvme5n17 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 157.6k 180.4k 0.0k 1.5M 1.8M 0.0k nvme5n18 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 146.4k 166.4k 0.0k 1.4M 1.6M 0.0k nvme5n19 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 237.6k 128.8k 0.0k 2.3M 1.3M 0.0k nvme5n2 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 184.8k 149.6k 0.0k 1.8M 1.5M 0.0k nvme5n20 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 145.2k 128.4k 0.0k 1.4M 1.3M 0.0k nvme5n3 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 143.2k 161.6k 0.0k 1.4M 1.6M 0.0k nvme5n4 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 219.6k 172.8k 0.0k 2.1M 1.7M 0.0k nvme5n5 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 169.2k 172.4k 0.0k 1.7M 1.7M 0.0k nvme5n6 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 186.8k 185.2k 0.0k 1.8M 1.8M 0.0k nvme5n7 2024-05-21T12:34:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 113.6k 202.4k 0.0k 1.1M 2.0M 0.0k nvme5n8 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 167.6k 180.0k 0.0k 1.6M 1.8M 0.0k nvme5n9 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 207.6k 146.8k 0.0k 2.0M 1.4M 0.0k nvme8n1 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 192.8k 209.2k 0.0k 1.9M 2.0M 0.0k nvme8n10 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 224.8k 157.6k 0.0k 2.2M 1.5M 0.0k nvme8n11 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 180.4k 160.8k 0.0k 1.8M 1.6M 0.0k nvme8n12 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 110.0k 201.2k 0.0k 1.1M 2.0M 0.0k nvme8n13 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 12.00 176.8k 186.8k 0.0k 1.7M 1.8M 0.0k nvme8n14 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 12.00 145.2k 195.2k 0.0k 1.4M 1.9M 0.0k nvme8n15 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 193.6k 225.2k 0.0k 1.9M 2.2M 0.0k nvme8n16 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 136.0k 150.0k 0.0k 1.3M 1.5M 0.0k nvme8n17 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 176.0k 177.6k 0.0k 1.7M 1.7M 0.0k nvme8n18 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 178.8k 172.8k 0.0k 1.7M 1.7M 0.0k nvme8n19 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 152.0k 209.6k 0.0k 1.5M 2.0M 0.0k nvme8n2 2024-05-21T12:34:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 234.0k 139.6k 0.0k 2.3M 1.4M 0.0k nvme8n20 2024-05-21T12:34:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 132.8k 184.4k 0.0k 1.3M 1.8M 0.0k nvme8n3 2024-05-21T12:34:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 179.6k 145.2k 0.0k 1.8M 1.4M 0.0k nvme8n4 2024-05-21T12:34:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 179.6k 160.4k 0.0k 1.8M 1.6M 0.0k nvme8n5 2024-05-21T12:34:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 166.4k 168.8k 0.0k 1.6M 1.6M 0.0k nvme8n6 2024-05-21T12:34:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 177.2k 166.4k 0.0k 1.7M 1.6M 0.0k nvme8n7 2024-05-21T12:34:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 162.0k 192.0k 0.0k 1.6M 1.9M 0.0k nvme8n8 2024-05-21T12:34:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.90 124.4k 141.6k 0.0k 1.2M 1.4M 0.0k nvme8n9 2024-05-21T12:34:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:04.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:04 smithi072 ceph-mon[33912]: pgmap v545: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.2 MiB/s rd, 9.1 MiB/s wr, 823 op/s 2024-05-21T12:34:05.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:04 smithi092 ceph-mon[33665]: pgmap v545: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.2 MiB/s rd, 9.1 MiB/s wr, 823 op/s 2024-05-21T12:34:05.059 INFO:teuthology.orchestra.run.smithi137.stdout: cluster: 2024-05-21T12:34:05.059 INFO:teuthology.orchestra.run.smithi137.stdout: id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:34:05.059 INFO:teuthology.orchestra.run.smithi137.stdout: health: HEALTH_OK 2024-05-21T12:34:05.059 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:34:05.059 INFO:teuthology.orchestra.run.smithi137.stdout: services: 2024-05-21T12:34:05.059 INFO:teuthology.orchestra.run.smithi137.stdout: mon: 3 daemons, quorum a,c,b (age 15m) 2024-05-21T12:34:05.059 INFO:teuthology.orchestra.run.smithi137.stdout: mgr: x(active, since 16m) 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: osd: 8 osds: 8 up (since 12m), 8 in (since 12m) 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: data: 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: pools: 2 pools, 33 pgs 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: objects: 485 objects, 1023 MiB 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: usage: 2.8 GiB used, 712 GiB / 715 GiB avail 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: pgs: 33 active+clean 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: io: 2024-05-21T12:34:05.060 INFO:teuthology.orchestra.run.smithi137.stdout: client: 9.2 MiB/s rd, 9.1 MiB/s wr, 419 op/s rd, 403 op/s wr 2024-05-21T12:34:05.061 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:34:05.089 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.107 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.126 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.145 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.164 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.219 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.237 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.256 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.274 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.293 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.311 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.330 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.349 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.367 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.386 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.404 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.422 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.441 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.459 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.478 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.496 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.514 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.533 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.551 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.569 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.588 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.607 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.625 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.643 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.662 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.681 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.700 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.719 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.737 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.755 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.774 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.792 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.811 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:05 smithi045 ceph-mon[30165]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:34:05.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:05 smithi045 ceph-mon[30165]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:34:05.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:05 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/798603377' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:34:05.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:05 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/59950504' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:34:05.829 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.848 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.866 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.885 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.904 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:05 smithi072 ceph-mon[33912]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:34:05.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:05 smithi072 ceph-mon[33912]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:34:05.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:05 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/798603377' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:34:05.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:05 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/59950504' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:34:05.922 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.941 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.959 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.978 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:05.996 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:05 smithi092 ceph-mon[33665]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:34:06.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:05 smithi092 ceph-mon[33665]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:34:06.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:05 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/798603377' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:34:06.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:05 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/59950504' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:34:06.014 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.033 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.052 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.070 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.088 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.107 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.126 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.144 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.163 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.181 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:34:06.182 INFO:tasks.nvmeof.[nvmeof.thrasher]:kill nvmeof.a 2024-05-21T12:34:06.182 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Stopping nvmeof.nvmeof.a... 2024-05-21T12:34:06.182 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:34:06.507 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:34:06 smithi045 systemd[1]: Stopping Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:34:06.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:34:06 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:34:06] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log.gz 2024-05-21T12:34:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:06 smithi045 ceph-mon[30165]: pgmap v546: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 735 op/s 2024-05-21T12:34:06.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:06 smithi072 ceph-mon[33912]: pgmap v546: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 735 op/s 2024-05-21T12:34:07.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:06 smithi092 ceph-mon[33665]: pgmap v546: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 735 op/s 2024-05-21T12:34:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:08 smithi045 ceph-mon[30165]: pgmap v547: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.9 MiB/s rd, 9.8 MiB/s wr, 750 op/s 2024-05-21T12:34:08.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:08 smithi072 ceph-mon[33912]: pgmap v547: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.9 MiB/s rd, 9.8 MiB/s wr, 750 op/s 2024-05-21T12:34:09.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:08 smithi092 ceph-mon[33665]: pgmap v547: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.9 MiB/s rd, 9.8 MiB/s wr, 750 op/s 2024-05-21T12:34:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:10 smithi045 ceph-mon[30165]: pgmap v548: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 689 op/s 2024-05-21T12:34:10.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:10 smithi072 ceph-mon[33912]: pgmap v548: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 689 op/s 2024-05-21T12:34:11.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:10 smithi092 ceph-mon[33665]: pgmap v548: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 689 op/s 2024-05-21T12:34:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:12 smithi045 ceph-mon[30165]: pgmap v549: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 710 op/s 2024-05-21T12:34:12.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:12 smithi072 ceph-mon[33912]: pgmap v549: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 710 op/s 2024-05-21T12:34:13.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:12 smithi092 ceph-mon[33665]: pgmap v549: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 710 op/s 2024-05-21T12:34:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:14 smithi045 ceph-mon[30165]: pgmap v550: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-05-21T12:34:14.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 216.8k 104.4k 0.0k 2.1M 1.0M 0.0k nvme2n1 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 195.6k 141.6k 0.0k 1.9M 1.4M 0.0k nvme2n10 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 158.0k 152.4k 0.0k 1.5M 1.5M 0.0k nvme2n11 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 162.4k 159.6k 0.0k 1.6M 1.6M 0.0k nvme2n12 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 151.6k 198.0k 0.0k 1.5M 1.9M 0.0k nvme2n13 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 156.0k 166.8k 0.0k 1.5M 1.6M 0.0k nvme2n14 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 169.6k 189.6k 0.0k 1.7M 1.9M 0.0k nvme2n15 2024-05-21T12:34:14.874 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 186.8k 143.2k 0.0k 1.8M 1.4M 0.0k nvme2n16 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 138.4k 204.4k 0.0k 1.4M 2.0M 0.0k nvme2n17 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 164.8k 152.0k 0.0k 1.6M 1.5M 0.0k nvme2n18 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 167.2k 195.2k 0.0k 1.6M 1.9M 0.0k nvme2n19 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 140.4k 172.0k 0.0k 1.4M 1.7M 0.0k nvme2n2 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 138.0k 165.2k 0.0k 1.3M 1.6M 0.0k nvme2n20 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 182.8k 178.0k 0.0k 1.8M 1.7M 0.0k nvme2n3 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 155.6k 188.8k 0.0k 1.5M 1.8M 0.0k nvme2n4 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 159.2k 173.2k 0.0k 1.6M 1.7M 0.0k nvme2n5 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 179.6k 192.4k 0.0k 1.8M 1.9M 0.0k nvme2n6 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 147.6k 206.0k 0.0k 1.4M 2.0M 0.0k nvme2n7 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 190.4k 166.0k 0.0k 1.9M 1.6M 0.0k nvme2n8 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 182.8k 157.6k 0.0k 1.8M 1.5M 0.0k nvme2n9 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 180.0k 158.0k 0.0k 1.8M 1.5M 0.0k nvme5n1 2024-05-21T12:34:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 185.6k 181.6k 0.0k 1.8M 1.8M 0.0k nvme5n10 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 176.0k 175.6k 0.0k 1.7M 1.7M 0.0k nvme5n11 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 159.6k 202.4k 0.0k 1.6M 2.0M 0.0k nvme5n12 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 165.2k 145.2k 0.0k 1.6M 1.4M 0.0k nvme5n13 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 200.4k 133.2k 0.0k 2.0M 1.3M 0.0k nvme5n14 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 166.4k 144.8k 0.0k 1.6M 1.4M 0.0k nvme5n15 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 162.0k 184.8k 0.0k 1.6M 1.8M 0.0k nvme5n16 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 136.0k 201.6k 0.0k 1.3M 2.0M 0.0k nvme5n17 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 192.4k 170.0k 0.0k 1.9M 1.7M 0.0k nvme5n18 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 170.0k 186.8k 0.0k 1.7M 1.8M 0.0k nvme5n19 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 166.8k 149.2k 0.0k 1.6M 1.5M 0.0k nvme5n2 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 161.6k 202.4k 0.0k 1.6M 2.0M 0.0k nvme5n20 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 181.2k 155.2k 0.0k 1.8M 1.5M 0.0k nvme5n3 2024-05-21T12:34:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 153.6k 178.8k 0.0k 1.5M 1.7M 0.0k nvme5n4 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 176.8k 141.2k 0.0k 1.7M 1.4M 0.0k nvme5n5 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 164.0k 134.4k 0.0k 1.6M 1.3M 0.0k nvme5n6 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 169.6k 173.2k 0.0k 1.7M 1.7M 0.0k nvme5n7 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 155.2k 158.4k 0.0k 1.5M 1.5M 0.0k nvme5n8 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 178.0k 173.6k 0.0k 1.7M 1.7M 0.0k nvme5n9 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 181.6k 164.4k 0.0k 1.8M 1.6M 0.0k nvme8n1 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 149.6k 177.2k 0.0k 1.5M 1.7M 0.0k nvme8n10 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 146.4k 171.6k 0.0k 1.4M 1.7M 0.0k nvme8n11 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 195.6k 160.4k 0.0k 1.9M 1.6M 0.0k nvme8n12 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 170.8k 184.8k 0.0k 1.7M 1.8M 0.0k nvme8n13 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 191.2k 149.2k 0.0k 1.9M 1.5M 0.0k nvme8n14 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 166.4k 158.8k 0.0k 1.6M 1.6M 0.0k nvme8n15 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 168.4k 138.4k 0.0k 1.6M 1.4M 0.0k nvme8n16 2024-05-21T12:34:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 194.4k 164.4k 0.0k 1.9M 1.6M 0.0k nvme8n17 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 132.4k 178.0k 0.0k 1.3M 1.7M 0.0k nvme8n18 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 155.6k 166.8k 0.0k 1.5M 1.6M 0.0k nvme8n19 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 140.8k 184.8k 0.0k 1.4M 1.8M 0.0k nvme8n2 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 192.0k 146.4k 0.0k 1.9M 1.4M 0.0k nvme8n20 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.80 188.8k 144.8k 0.0k 1.8M 1.4M 0.0k nvme8n3 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 180.8k 182.4k 0.0k 1.8M 1.8M 0.0k nvme8n4 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 174.4k 174.4k 0.0k 1.7M 1.7M 0.0k nvme8n5 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 178.0k 172.4k 0.0k 1.7M 1.7M 0.0k nvme8n6 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 153.2k 180.4k 0.0k 1.5M 1.8M 0.0k nvme8n7 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 194.4k 143.2k 0.0k 1.9M 1.4M 0.0k nvme8n8 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.70 188.8k 188.0k 0.0k 1.8M 1.8M 0.0k nvme8n9 2024-05-21T12:34:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:14.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:14 smithi072 ceph-mon[33912]: pgmap v550: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-05-21T12:34:15.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:14 smithi092 ceph-mon[33665]: pgmap v550: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-05-21T12:34:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:16 smithi045 ceph-mon[30165]: pgmap v551: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 721 op/s 2024-05-21T12:34:16.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:34:16 smithi045 bash[74384]: time="2024-05-21T12:34:16Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a in 10 seconds, resorting to SIGKILL" 2024-05-21T12:34:16.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:34:16 smithi045 podman[74384]: 2024-05-21 12:34:16.532729287 +0000 UTC m=+10.136799303 container died c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, architecture=x86_64, io.ceph.component=ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, org.centos.version=9.0-21.el9, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.version=1.2.9, io.spdk.build.ceph-release=19.0.0, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, com.redhat.component=ubi9-container, io.spdk.git.repo=https://github.com/ceph/spdk.git, vcs-type=git, io.spdk.build.makeflags=, io.openshift.wants=ceph,rbd, io.buildah.version=1.29.0, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.target-arch=, io.spdk.build.pkgdep-args=--rbd, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, url=https://github.com/ceph/ceph-nvmeof, version=1.2.9, build-date=2024-05-19 17:57:19 UTC, distribution-scope=public, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.version=24.01, vendor=Red Hat, Inc., release=, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., summary=Ceph NVMe over Fabrics Gateway, io.ceph.summary=Ceph NVMe over Fabrics Gateway, name=ceph-nvmeof, io.spdk.git.branch=undefined, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.maintainer=Ceph Developers , io.ceph.nvmeof=None) 2024-05-21T12:34:16.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:16 smithi072 ceph-mon[33912]: pgmap v551: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 721 op/s 2024-05-21T12:34:17.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:16 smithi092 ceph-mon[33665]: pgmap v551: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 721 op/s 2024-05-21T12:34:17.469 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:34:17 smithi045 podman[74384]: 2024-05-21 12:34:17.18461411 +0000 UTC m=+10.788684123 container cleanup c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, summary=Ceph NVMe over Fabrics Gateway, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.maintainer=Ceph Developers , io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.version=1.2.9, vendor=Red Hat, Inc., io.ceph.git.branch=tags/1.2.9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.pkgdep-args=--rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.version=24.01, url=https://github.com/ceph/ceph-nvmeof, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., version=1.2.9, org.centos.version=9.0-21.el9, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, io.ceph.component=ceph-nvmeof, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, build-date=2024-05-19 17:57:19 UTC, io.openshift.wants=ceph,rbd, io.ceph.nvmeof=None, io.buildah.version=1.29.0, architecture=x86_64, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, release=, io.spdk.git.branch=undefined, io.spdk.build.ceph-release=19.0.0, com.redhat.component=ubi9-container, maintainer=Ceph Developers , name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-type=git, distribution-scope=public, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/) 2024-05-21T12:34:17.469 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:34:17 smithi045 bash[74384]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a 2024-05-21T12:34:17.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service' 2024-05-21T12:34:17.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:34:17.653 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Stopped nvmeof.nvmeof.a 2024-05-21T12:34:17.653 INFO:tasks.nvmeof.[nvmeof.thrasher]:kill nvmeof.b 2024-05-21T12:34:17.653 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.b:Stopping nvmeof.nvmeof.b... 2024-05-21T12:34:17.653 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:34:17.989 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:17 smithi072 systemd[1]: Stopping Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:34:18.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:34:17] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.b/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.b/nvmeof-log.gz 2024-05-21T12:34:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:18 smithi045 ceph-mon[30165]: pgmap v552: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 736 op/s 2024-05-21T12:34:18.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:18 smithi072 ceph-mon[33912]: pgmap v552: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 736 op/s 2024-05-21T12:34:19.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:18 smithi092 ceph-mon[33665]: pgmap v552: 33 pgs: 33 active+clean; 1023 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 10 MiB/s wr, 736 op/s 2024-05-21T12:34:20.804 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:20 smithi092 ceph-mon[33665]: pgmap v553: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 8.7 MiB/s rd, 8.9 MiB/s wr, 664 op/s 2024-05-21T12:34:20.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:20 smithi045 ceph-mon[30165]: pgmap v553: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 8.7 MiB/s rd, 8.9 MiB/s wr, 664 op/s 2024-05-21T12:34:20.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:20 smithi072 ceph-mon[33912]: pgmap v553: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 8.7 MiB/s rd, 8.9 MiB/s wr, 664 op/s 2024-05-21T12:34:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:22 smithi045 ceph-mon[30165]: pgmap v554: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 7.9 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-05-21T12:34:22.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:22 smithi072 ceph-mon[33912]: pgmap v554: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 7.9 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-05-21T12:34:23.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:22 smithi092 ceph-mon[33665]: pgmap v554: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 7.9 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2024-05-21T12:34:24.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:24 smithi045 ceph-mon[30165]: pgmap v555: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-05-21T12:34:24.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:34:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 24.4k 20.0k 0.0k 244.0k 200.0k 0.0k nvme2n1 2024-05-21T12:34:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 18.8k 30.0k 0.0k 188.0k 300.0k 0.0k nvme2n10 2024-05-21T12:34:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 14.4k 38.4k 0.0k 144.0k 384.0k 0.0k nvme2n11 2024-05-21T12:34:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 24.8k 29.6k 0.0k 248.0k 296.0k 0.0k nvme2n12 2024-05-21T12:34:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 28.0k 27.2k 0.0k 280.0k 272.0k 0.0k nvme2n13 2024-05-21T12:34:24.875 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 26.8k 30.0k 0.0k 268.0k 300.0k 0.0k nvme2n14 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 24.8k 18.4k 0.0k 248.0k 184.0k 0.0k nvme2n15 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 30.8k 18.4k 0.0k 308.0k 184.0k 0.0k nvme2n16 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 22.0k 18.8k 0.0k 220.0k 188.0k 0.0k nvme2n17 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 11.2k 34.0k 0.0k 112.0k 340.0k 0.0k nvme2n18 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 47.2k 15.2k 0.0k 472.0k 152.0k 0.0k nvme2n19 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 29.6k 34.0k 0.0k 296.0k 340.0k 0.0k nvme2n2 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 34.0k 18.4k 0.0k 340.0k 184.0k 0.0k nvme2n20 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 33.2k 18.0k 0.0k 332.0k 180.0k 0.0k nvme2n3 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 27.2k 18.4k 0.0k 272.0k 184.0k 0.0k nvme2n4 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 25.6k 19.2k 0.0k 256.0k 192.0k 0.0k nvme2n5 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 27.2k 26.8k 0.0k 272.0k 268.0k 0.0k nvme2n6 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 22.8k 27.2k 0.0k 228.0k 272.0k 0.0k nvme2n7 2024-05-21T12:34:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 14.0k 40.4k 0.0k 140.0k 404.0k 0.0k nvme2n8 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 26.8k 26.4k 0.0k 268.0k 264.0k 0.0k nvme2n9 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 35.2k 29.2k 0.0k 352.0k 292.0k 0.0k nvme5n1 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 18.0k 27.6k 0.0k 180.0k 276.0k 0.0k nvme5n10 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 27.2k 21.2k 0.0k 272.0k 212.0k 0.0k nvme5n11 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 31.6k 24.0k 0.0k 316.0k 240.0k 0.0k nvme5n12 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 33.6k 21.2k 0.0k 336.0k 212.0k 0.0k nvme5n13 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 30.4k 30.4k 0.0k 304.0k 304.0k 0.0k nvme5n14 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 20.8k 23.6k 0.0k 208.0k 236.0k 0.0k nvme5n15 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 22.4k 41.2k 0.0k 224.0k 412.0k 0.0k nvme5n16 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 20.8k 24.0k 0.0k 208.0k 240.0k 0.0k nvme5n17 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 18.8k 26.0k 0.0k 188.0k 260.0k 0.0k nvme5n18 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 9.6k 38.0k 0.0k 96.0k 380.0k 0.0k nvme5n19 2024-05-21T12:34:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 30.4k 22.8k 0.0k 304.0k 228.0k 0.0k nvme5n2 2024-05-21T12:34:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 12.0k 35.2k 0.0k 120.0k 352.0k 0.0k nvme5n20 2024-05-21T12:34:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 24.8k 26.8k 0.0k 248.0k 268.0k 0.0k nvme5n3 2024-05-21T12:34:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 16.8k 30.0k 0.0k 168.0k 300.0k 0.0k nvme5n4 2024-05-21T12:34:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 14.4k 29.2k 0.0k 144.0k 292.0k 0.0k nvme5n5 2024-05-21T12:34:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 42.0k 8.8k 0.0k 420.0k 88.0k 0.0k nvme5n6 2024-05-21T12:34:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 18.0k 36.0k 0.0k 180.0k 360.0k 0.0k nvme5n7 2024-05-21T12:34:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 36.0k 24.0k 0.0k 360.0k 240.0k 0.0k nvme5n8 2024-05-21T12:34:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 27.6k 23.6k 0.0k 276.0k 236.0k 0.0k nvme5n9 2024-05-21T12:34:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 23.2k 34.8k 0.0k 232.0k 348.0k 0.0k nvme8n1 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 43.2k 7.6k 0.0k 432.0k 76.0k 0.0k nvme8n10 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 26.4k 20.0k 0.0k 264.0k 200.0k 0.0k nvme8n11 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 45.6k 18.8k 0.0k 456.0k 188.0k 0.0k nvme8n12 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 14.8k 14.8k 0.0k 148.0k 148.0k 0.0k nvme8n13 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 30.8k 10.4k 0.0k 308.0k 104.0k 0.0k nvme8n14 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 15.6k 35.2k 0.0k 156.0k 352.0k 0.0k nvme8n15 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 9.6k 38.0k 0.0k 96.0k 380.0k 0.0k nvme8n16 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 16.0k 36.4k 0.0k 160.0k 364.0k 0.0k nvme8n17 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 29.6k 28.0k 0.0k 296.0k 280.0k 0.0k nvme8n18 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 9.6k 29.2k 0.0k 96.0k 292.0k 0.0k nvme8n19 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 20.0k 18.0k 0.0k 200.0k 180.0k 0.0k nvme8n2 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 16.8k 32.0k 0.0k 168.0k 320.0k 0.0k nvme8n20 2024-05-21T12:34:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 20.4k 28.0k 0.0k 204.0k 280.0k 0.0k nvme8n3 2024-05-21T12:34:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 25.2k 22.4k 0.0k 252.0k 224.0k 0.0k nvme8n4 2024-05-21T12:34:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 34.4k 31.6k 0.0k 344.0k 316.0k 0.0k nvme8n5 2024-05-21T12:34:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 26.0k 13.2k 0.0k 260.0k 132.0k 0.0k nvme8n6 2024-05-21T12:34:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 34.8k 25.6k 0.0k 348.0k 256.0k 0.0k nvme8n7 2024-05-21T12:34:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2.00 26.8k 38.0k 0.0k 268.0k 380.0k 0.0k nvme8n8 2024-05-21T12:34:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 1.90 13.2k 22.8k 0.0k 132.0k 228.0k 0.0k nvme8n9 2024-05-21T12:34:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:24.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:24 smithi072 ceph-mon[33912]: pgmap v555: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-05-21T12:34:25.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:24 smithi092 ceph-mon[33665]: pgmap v555: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 508 op/s 2024-05-21T12:34:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:26 smithi045 ceph-mon[30165]: pgmap v556: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 328 op/s 2024-05-21T12:34:26.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:26 smithi072 ceph-mon[33912]: pgmap v556: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 328 op/s 2024-05-21T12:34:27.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:26 smithi092 ceph-mon[33665]: pgmap v556: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 328 op/s 2024-05-21T12:34:28.291 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:27 smithi072 bash[56230]: time="2024-05-21T12:34:27Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b in 10 seconds, resorting to SIGKILL" 2024-05-21T12:34:28.291 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:28 smithi072 podman[56230]: 2024-05-21 12:34:28.006771742 +0000 UTC m=+10.142767636 container died a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.version=24.01, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, com.redhat.component=ubi9-container, version=1.2.9, summary=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.git.branch=undefined, io.spdk.build.target-arch=, url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.build.ceph-release=19.0.0, io.ceph.nvmeof=None, vendor=Red Hat, Inc., io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, name=ceph-nvmeof, build-date=2024-05-19 17:57:19 UTC, io.buildah.version=1.29.0, io.ceph.summary=Ceph NVMe over Fabrics Gateway, maintainer=Ceph Developers , io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., architecture=x86_64, release=, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.maintainer=Ceph Developers , org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, distribution-scope=public, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.pkgdep-args=--rbd, io.ceph.component=ceph-nvmeof, vcs-type=git, io.ceph.url=https://github.com/ceph/ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.makeflags=, io.ceph.version=1.2.9, org.centos.version=9.0-21.el9, io.ceph.git.branch=tags/1.2.9) 2024-05-21T12:34:28.619 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:28 smithi072 ceph-mon[33912]: pgmap v557: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-05-21T12:34:28.619 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:28 smithi072 ceph-mon[33912]: osdmap e58: 8 total, 8 up, 8 in 2024-05-21T12:34:28.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:28 smithi072 podman[56230]: 2024-05-21 12:34:28.450610165 +0000 UTC m=+10.586606101 container cleanup a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.spdk.build.pkgdep-args=--rbd, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, maintainer=Ceph Developers , io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.nvmeof=None, io.openshift.wants=ceph,rbd, io.spdk.build.target-arch=, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, vcs-type=git, org.centos.version=9.0-21.el9, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.url=https://github.com/ceph/ceph-nvmeof, build-date=2024-05-19 17:57:19 UTC, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, vendor=Red Hat, Inc., description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.buildah.version=1.29.0, io.spdk.build.ceph-release=19.0.0, summary=Ceph NVMe over Fabrics Gateway, distribution-scope=public, io.spdk.git.branch=undefined, io.spdk.version=24.01, com.redhat.component=ubi9-container, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, architecture=x86_64, io.ceph.version=1.2.9, io.ceph.maintainer=Ceph Developers , io.ceph.component=ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://github.com/ceph/ceph-nvmeof, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.git.repo=https://github.com/ceph/spdk.git, release=, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, version=1.2.9) 2024-05-21T12:34:28.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:28 smithi072 bash[56230]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b 2024-05-21T12:34:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:28 smithi045 ceph-mon[30165]: pgmap v557: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-05-21T12:34:28.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:28 smithi045 ceph-mon[30165]: osdmap e58: 8 total, 8 up, 8 in 2024-05-21T12:34:28.849 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service' 2024-05-21T12:34:28.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:28 smithi072 podman[56258]: 2024-05-21 12:34:28.620056554 +0000 UTC m=+0.612793502 container remove a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.ceph.git.branch=tags/1.2.9, vcs-type=git, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.makeflags=, io.spdk.version=24.01, io.ceph.maintainer=Ceph Developers , release=, version=1.2.9, io.buildah.version=1.29.0, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.nvmeof=None, io.ceph.version=1.2.9, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.build.ceph-release=19.0.0, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., summary=Ceph NVMe over Fabrics Gateway, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, com.redhat.component=ubi9-container, vendor=Red Hat, Inc., io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.git.repo=https://github.com/ceph/spdk.git, distribution-scope=public, org.centos.version=9.0-21.el9, io.spdk.build.target-arch=, architecture=x86_64, io.spdk.git.branch=undefined, maintainer=Ceph Developers , io.ceph.component=ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.summary=Ceph NVMe over Fabrics Gateway, name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.wants=ceph,rbd, url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, build-date=2024-05-19 17:57:19 UTC, io.ceph.url=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:34:28.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:28 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:34:28.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:28 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Failed with result 'exit-code'. 2024-05-21T12:34:28.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:28 smithi072 systemd[1]: Stopped Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:34:28.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:34:28 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Consumed 10min 18.625s CPU time. 2024-05-21T12:34:28.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:34:28.906 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.b:Stopped nvmeof.nvmeof.b 2024-05-21T12:34:28.906 INFO:tasks.nvmeof.[nvmeof.thrasher]:waiting for 100 secs before reviving 2024-05-21T12:34:29.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:28 smithi092 ceph-mon[33665]: pgmap v557: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-05-21T12:34:29.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:28 smithi092 ceph-mon[33665]: osdmap e58: 8 total, 8 up, 8 in 2024-05-21T12:34:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:30 smithi045 ceph-mon[30165]: pgmap v559: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 505 KiB/s rd, 614 KiB/s wr, 42 op/s 2024-05-21T12:34:30.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:30 smithi072 ceph-mon[33912]: pgmap v559: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 505 KiB/s rd, 614 KiB/s wr, 42 op/s 2024-05-21T12:34:31.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:30 smithi092 ceph-mon[33665]: pgmap v559: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 505 KiB/s rd, 614 KiB/s wr, 42 op/s 2024-05-21T12:34:32.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:32 smithi045 ceph-mon[30165]: pgmap v560: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:34:32.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:32 smithi072 ceph-mon[33912]: pgmap v560: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:34:33.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:32 smithi092 ceph-mon[33665]: pgmap v560: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:34:34.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:34 smithi045 ceph-mon[30165]: pgmap v561: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 14 KiB/s rd, 1 op/s 2024-05-21T12:34:34.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n1 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n10 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n11 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n12 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n13 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n14 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n15 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n16 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n17 2024-05-21T12:34:34.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n18 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n19 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n2 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n20 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n3 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n4 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n5 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n6 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n7 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n8 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n9 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n1 2024-05-21T12:34:34.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n10 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n11 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n12 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n13 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n14 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n15 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n16 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n17 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n18 2024-05-21T12:34:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n19 2024-05-21T12:34:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n2 2024-05-21T12:34:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n20 2024-05-21T12:34:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n3 2024-05-21T12:34:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n4 2024-05-21T12:34:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n5 2024-05-21T12:34:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n6 2024-05-21T12:34:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n7 2024-05-21T12:34:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n8 2024-05-21T12:34:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:34:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n1 2024-05-21T12:34:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n10 2024-05-21T12:34:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n11 2024-05-21T12:34:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n12 2024-05-21T12:34:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n13 2024-05-21T12:34:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n14 2024-05-21T12:34:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n15 2024-05-21T12:34:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n16 2024-05-21T12:34:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n17 2024-05-21T12:34:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n18 2024-05-21T12:34:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n19 2024-05-21T12:34:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n2 2024-05-21T12:34:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n20 2024-05-21T12:34:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n3 2024-05-21T12:34:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n4 2024-05-21T12:34:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n5 2024-05-21T12:34:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n6 2024-05-21T12:34:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n7 2024-05-21T12:34:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:34:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n9 2024-05-21T12:34:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:34.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:34 smithi072 ceph-mon[33912]: pgmap v561: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 14 KiB/s rd, 1 op/s 2024-05-21T12:34:35.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:34 smithi092 ceph-mon[33665]: pgmap v561: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 14 KiB/s rd, 1 op/s 2024-05-21T12:34:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:36 smithi045 ceph-mon[30165]: pgmap v562: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 14 KiB/s rd, 1 op/s 2024-05-21T12:34:36.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:36 smithi072 ceph-mon[33912]: pgmap v562: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 14 KiB/s rd, 1 op/s 2024-05-21T12:34:37.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:36 smithi092 ceph-mon[33665]: pgmap v562: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 14 KiB/s rd, 1 op/s 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:34:38] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:38.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:34:38.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:38.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:34:38.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:38.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:38.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:51526 2024-05-21T12:34:38.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:34:38.034165] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_2_0 elapsed time: 0.000296 seconds 2024-05-21T12:34:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:38 smithi045 ceph-mon[30165]: pgmap v563: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:38 smithi045 ceph-mon[30165]: osdmap e59: 8 total, 8 up, 8 in 2024-05-21T12:34:38.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:38 smithi072 ceph-mon[33912]: pgmap v563: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:38.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:38 smithi072 ceph-mon[33912]: osdmap e59: 8 total, 8 up, 8 in 2024-05-21T12:34:39.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-mon[33665]: pgmap v563: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:39.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:38 smithi092 ceph-mon[33665]: osdmap e59: 8 total, 8 up, 8 in 2024-05-21T12:34:40.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:40 smithi045 ceph-mon[30165]: pgmap v565: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:40.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:40 smithi072 ceph-mon[33912]: pgmap v565: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:41.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-mon[33665]: pgmap v565: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:41.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:34:40] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:34:41.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:41.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:41.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:51540 2024-05-21T12:34:41.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:34:40.554275] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_2_0 elapsed time: 0.000251 seconds 2024-05-21T12:34:42.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:42 smithi045 ceph-mon[30165]: pgmap v566: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:42.915 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:42 smithi072 ceph-mon[33912]: pgmap v566: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:43.002 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:42 smithi092 ceph-mon[33665]: pgmap v566: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.2 KiB/s rd, 0 op/s 2024-05-21T12:34:43.502 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:34:43] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:34:43.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:43.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:34:43.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:43.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:43.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:34:43.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:43.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:34:43.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:43.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:43.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:34:43.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:43.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:34:43.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:43.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:43.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:51560 2024-05-21T12:34:43.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:34:43.034556] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000308 seconds 2024-05-21T12:34:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:44 smithi045 ceph-mon[30165]: osdmap e60: 8 total, 8 up, 8 in 2024-05-21T12:34:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:44 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:44 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:44 smithi045 ceph-mon[30165]: pgmap v568: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 0 op/s 2024-05-21T12:34:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:44 smithi072 ceph-mon[33912]: osdmap e60: 8 total, 8 up, 8 in 2024-05-21T12:34:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:44 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:44 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:44 smithi072 ceph-mon[33912]: pgmap v568: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 0 op/s 2024-05-21T12:34:44.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-mon[33665]: osdmap e60: 8 total, 8 up, 8 in 2024-05-21T12:34:44.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:44.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:44.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-mon[33665]: pgmap v568: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 0 op/s 2024-05-21T12:34:44.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:34:44] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:34:44.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:44.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:34:44.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:44.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:44.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:34:44.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:44.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:48746 2024-05-21T12:34:44.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:34:44.042504] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000202 seconds 2024-05-21T12:34:44.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:34:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n1 2024-05-21T12:34:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n10 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n11 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n12 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n13 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n14 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n15 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n16 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n17 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n18 2024-05-21T12:34:44.885 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n19 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n2 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n20 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n3 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n4 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n5 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n6 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n7 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n8 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme2n9 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n1 2024-05-21T12:34:44.886 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme5n10 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n11 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n12 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n13 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n14 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n15 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n16 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n17 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n18 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n19 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n2 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n20 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n3 2024-05-21T12:34:44.887 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n4 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n5 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n6 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n7 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n8 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n1 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n10 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n11 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n12 2024-05-21T12:34:44.888 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n13 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n14 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n15 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n16 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n17 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n18 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n19 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n2 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n20 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n3 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n4 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n5 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n6 2024-05-21T12:34:44.889 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n7 2024-05-21T12:34:44.890 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:34:44.890 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme8n9 2024-05-21T12:34:44.890 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:45 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:45 smithi045 ceph-mon[30165]: osdmap e61: 8 total, 8 up, 8 in 2024-05-21T12:34:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:45 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:45 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:45 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:45 smithi072 ceph-mon[33912]: osdmap e61: 8 total, 8 up, 8 in 2024-05-21T12:34:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:45 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:45 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:45.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:45 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:45.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:45 smithi092 ceph-mon[33665]: osdmap e61: 8 total, 8 up, 8 in 2024-05-21T12:34:45.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:45 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:45.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:45 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:46.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:46 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:46.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:46 smithi045 ceph-mon[30165]: osdmap e62: 8 total, 8 up, 8 in 2024-05-21T12:34:46.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:46 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:46.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:46 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:46.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:46 smithi045 ceph-mon[30165]: pgmap v571: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-21T12:34:46.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:46 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:46.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:46 smithi072 ceph-mon[33912]: osdmap e62: 8 total, 8 up, 8 in 2024-05-21T12:34:46.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:46 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:46.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:46 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:46.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:46 smithi072 ceph-mon[33912]: pgmap v571: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-21T12:34:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:46 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:46 smithi092 ceph-mon[33665]: osdmap e62: 8 total, 8 up, 8 in 2024-05-21T12:34:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:46 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:46 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:46 smithi092 ceph-mon[33665]: pgmap v571: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 9.0 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-21T12:34:47.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:47 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:47.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:47 smithi045 ceph-mon[30165]: osdmap e63: 8 total, 8 up, 8 in 2024-05-21T12:34:47.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:47 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:47.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:47 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:47.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:47 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:47.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:47 smithi072 ceph-mon[33912]: osdmap e63: 8 total, 8 up, 8 in 2024-05-21T12:34:47.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:47 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:47.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:47 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:47.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:47 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:47.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:47 smithi092 ceph-mon[33665]: osdmap e63: 8 total, 8 up, 8 in 2024-05-21T12:34:47.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:47 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:47.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:47 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:48 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:48 smithi045 ceph-mon[30165]: osdmap e64: 8 total, 8 up, 8 in 2024-05-21T12:34:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:48 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:48 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:48 smithi045 ceph-mon[30165]: pgmap v574: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 18 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-05-21T12:34:48.317 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:48 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:48.317 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:48 smithi072 ceph-mon[33912]: osdmap e64: 8 total, 8 up, 8 in 2024-05-21T12:34:48.317 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:48 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:48.317 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:48 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:48.317 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:48 smithi072 ceph-mon[33912]: pgmap v574: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 18 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-05-21T12:34:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:48 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:48 smithi092 ceph-mon[33665]: osdmap e64: 8 total, 8 up, 8 in 2024-05-21T12:34:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:48 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:48.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:48 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:48.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:48 smithi092 ceph-mon[33665]: pgmap v574: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 18 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-05-21T12:34:49.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:49 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:49.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:49 smithi045 ceph-mon[30165]: osdmap e65: 8 total, 8 up, 8 in 2024-05-21T12:34:49.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:49 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:49.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:49 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:49.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:49 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:49.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:49 smithi072 ceph-mon[33912]: osdmap e65: 8 total, 8 up, 8 in 2024-05-21T12:34:49.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:49 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:49.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:49 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:49.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:49 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:49.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:49 smithi092 ceph-mon[33665]: osdmap e65: 8 total, 8 up, 8 in 2024-05-21T12:34:49.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:49 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:49.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:49 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:50.403 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:50 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:50.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:50 smithi092 ceph-mon[33665]: osdmap e66: 8 total, 8 up, 8 in 2024-05-21T12:34:50.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:50 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:50.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:50 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:50.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:50 smithi092 ceph-mon[33665]: pgmap v577: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-05-21T12:34:50.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:50 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:50.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:50 smithi072 ceph-mon[33912]: osdmap e66: 8 total, 8 up, 8 in 2024-05-21T12:34:50.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:50 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:50.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:50 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:50.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:50 smithi072 ceph-mon[33912]: pgmap v577: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-05-21T12:34:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:50 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:50 smithi045 ceph-mon[30165]: osdmap e66: 8 total, 8 up, 8 in 2024-05-21T12:34:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:50 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:50 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:50 smithi045 ceph-mon[30165]: pgmap v577: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-05-21T12:34:51.360 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:51 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:51.360 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:51 smithi072 ceph-mon[33912]: osdmap e67: 8 total, 8 up, 8 in 2024-05-21T12:34:51.360 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:51 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:51.360 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:51 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:51.361 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:51 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:34:51.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:51 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:51.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:51 smithi092 ceph-mon[33665]: osdmap e67: 8 total, 8 up, 8 in 2024-05-21T12:34:51.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:51 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:51.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:51 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:51.374 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:51 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:34:51.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:51 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:51.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:51 smithi045 ceph-mon[30165]: osdmap e67: 8 total, 8 up, 8 in 2024-05-21T12:34:51.381 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:51 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:51.381 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:51 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:51.381 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:51 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:34:52.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:52 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:52.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:52 smithi045 ceph-mon[30165]: osdmap e68: 8 total, 8 up, 8 in 2024-05-21T12:34:52.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:52 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:52.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:52 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:52.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:52 smithi045 ceph-mon[30165]: pgmap v580: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 45 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-05-21T12:34:52.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:52 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:52.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:52 smithi072 ceph-mon[33912]: osdmap e68: 8 total, 8 up, 8 in 2024-05-21T12:34:52.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:52 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:52.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:52 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:52.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:52 smithi072 ceph-mon[33912]: pgmap v580: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 45 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-05-21T12:34:52.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:52 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:52.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:52 smithi092 ceph-mon[33665]: osdmap e68: 8 total, 8 up, 8 in 2024-05-21T12:34:52.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:52 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:52.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:52 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:52.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:52 smithi092 ceph-mon[33665]: pgmap v580: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 45 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-05-21T12:34:53.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: osdmap e69: 8 total, 8 up, 8 in 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:34:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:34:53.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:53 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: osdmap e69: 8 total, 8 up, 8 in 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:34:53.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:53 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:53.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: osdmap e69: 8 total, 8 up, 8 in 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:34:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:53 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:34:54.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:54 smithi072 ceph-mon[33912]: pgmap v582: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-05-21T12:34:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:54 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:54 smithi072 ceph-mon[33912]: osdmap e70: 8 total, 8 up, 8 in 2024-05-21T12:34:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:54 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:54 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:54 smithi072 ceph-mon[33912]: Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-21T12:34:54.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:54 smithi092 ceph-mon[33665]: pgmap v582: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-05-21T12:34:54.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:54 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:54.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:54 smithi092 ceph-mon[33665]: osdmap e70: 8 total, 8 up, 8 in 2024-05-21T12:34:54.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:54 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:54.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:54 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:54.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:54 smithi092 ceph-mon[33665]: Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-21T12:34:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:54 smithi045 ceph-mon[30165]: pgmap v582: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 69 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-05-21T12:34:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:54 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:54.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:54 smithi045 ceph-mon[30165]: osdmap e70: 8 total, 8 up, 8 in 2024-05-21T12:34:54.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:54 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:54.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:54 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:54.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:54 smithi045 ceph-mon[30165]: Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-21T12:34:54.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:34:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n1 2024-05-21T12:34:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme2n10 2024-05-21T12:34:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme2n11 2024-05-21T12:34:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.6k 0.0k 0.0k 56.0k 0.0k 0.0k nvme2n12 2024-05-21T12:34:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme2n13 2024-05-21T12:34:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme2n14 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.4k 0.0k 0.0k 4.0k 0.0k nvme2n15 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme2n16 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n17 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme2n18 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n19 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n2 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n20 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n3 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n4 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n5 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n6 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n7 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n8 2024-05-21T12:34:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n9 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n1 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n10 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme5n11 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.4k 0.0k 0.0k 44.0k 0.0k nvme5n12 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme5n13 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme5n14 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.4k 0.0k 0.0k 4.0k 0.0k nvme5n15 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme5n16 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme5n17 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme5n18 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n19 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n2 2024-05-21T12:34:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n20 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n3 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n4 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n5 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n6 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n7 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n8 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n1 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme8n10 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.2k 0.0k 0.0k 12.0k 0.0k nvme8n11 2024-05-21T12:34:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.8k 0.0k 0.0k 48.0k 0.0k 0.0k nvme8n12 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.4k 0.0k 0.0k 44.0k 0.0k nvme8n13 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme8n14 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme8n15 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme8n16 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.4k 0.0k 0.0k 44.0k 0.0k nvme8n17 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n18 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n19 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n2 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n20 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n3 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n4 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n5 2024-05-21T12:34:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n6 2024-05-21T12:34:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n7 2024-05-21T12:34:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:34:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n9 2024-05-21T12:34:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:34:55.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:55 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:55.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:55 smithi072 ceph-mon[33912]: osdmap e71: 8 total, 8 up, 8 in 2024-05-21T12:34:55.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:55 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:55.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:55 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:55.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:55 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:55.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:55 smithi092 ceph-mon[33665]: osdmap e71: 8 total, 8 up, 8 in 2024-05-21T12:34:55.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:55 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:55.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:55 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:55 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:55 smithi045 ceph-mon[30165]: osdmap e71: 8 total, 8 up, 8 in 2024-05-21T12:34:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:55 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:55 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:56 smithi072 ceph-mon[33912]: pgmap v585: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 72 KiB/s wr, 29 op/s 2024-05-21T12:34:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:56 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:56 smithi072 ceph-mon[33912]: osdmap e72: 8 total, 8 up, 8 in 2024-05-21T12:34:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:56 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:56 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:56 smithi092 ceph-mon[33665]: pgmap v585: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 72 KiB/s wr, 29 op/s 2024-05-21T12:34:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:56 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:56 smithi092 ceph-mon[33665]: osdmap e72: 8 total, 8 up, 8 in 2024-05-21T12:34:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:56 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:56 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:56 smithi045 ceph-mon[30165]: pgmap v585: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 36 KiB/s rd, 72 KiB/s wr, 29 op/s 2024-05-21T12:34:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:56 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:56 smithi045 ceph-mon[30165]: osdmap e72: 8 total, 8 up, 8 in 2024-05-21T12:34:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:56 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:56 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:57.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:57 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:57.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:57 smithi072 ceph-mon[33912]: osdmap e73: 8 total, 8 up, 8 in 2024-05-21T12:34:57.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:57 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:57.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:57 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:57.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:57 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:57.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:57 smithi092 ceph-mon[33665]: osdmap e73: 8 total, 8 up, 8 in 2024-05-21T12:34:57.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:57 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:57.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:57 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:57 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:34:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:57 smithi045 ceph-mon[30165]: osdmap e73: 8 total, 8 up, 8 in 2024-05-21T12:34:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:57 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:57.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:57 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:58 smithi072 ceph-mon[33912]: pgmap v588: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 26 op/s 2024-05-21T12:34:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:58 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:58 smithi072 ceph-mon[33912]: osdmap e74: 8 total, 8 up, 8 in 2024-05-21T12:34:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:58 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:58 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:58 smithi092 ceph-mon[33665]: pgmap v588: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 26 op/s 2024-05-21T12:34:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:58 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:58 smithi092 ceph-mon[33665]: osdmap e74: 8 total, 8 up, 8 in 2024-05-21T12:34:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:58 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:58 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:58 smithi045 ceph-mon[30165]: pgmap v588: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 40 KiB/s wr, 26 op/s 2024-05-21T12:34:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:58 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:58 smithi045 ceph-mon[30165]: osdmap e74: 8 total, 8 up, 8 in 2024-05-21T12:34:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:58 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:58.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:58 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:34:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:59 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:59 smithi072 ceph-mon[33912]: osdmap e75: 8 total, 8 up, 8 in 2024-05-21T12:34:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:59 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:34:59 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:59.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:59 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:59.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:59 smithi092 ceph-mon[33665]: osdmap e75: 8 total, 8 up, 8 in 2024-05-21T12:34:59.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:59 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:59.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:34:59 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:59 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:34:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:59 smithi045 ceph-mon[30165]: osdmap e75: 8 total, 8 up, 8 in 2024-05-21T12:34:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:59 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:34:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:34:59 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:00 smithi072 ceph-mon[33912]: pgmap v591: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-05-21T12:35:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:00 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:00 smithi072 ceph-mon[33912]: osdmap e76: 8 total, 8 up, 8 in 2024-05-21T12:35:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:00 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:00 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:00 smithi092 ceph-mon[33665]: pgmap v591: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-05-21T12:35:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:00 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:00 smithi092 ceph-mon[33665]: osdmap e76: 8 total, 8 up, 8 in 2024-05-21T12:35:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:00 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:00 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:00 smithi045 ceph-mon[30165]: pgmap v591: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-05-21T12:35:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:00 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:00 smithi045 ceph-mon[30165]: osdmap e76: 8 total, 8 up, 8 in 2024-05-21T12:35:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:00 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:00 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:01 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:01 smithi072 ceph-mon[33912]: osdmap e77: 8 total, 8 up, 8 in 2024-05-21T12:35:01.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:01 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:01.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:01 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:01.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:01 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:01.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:01 smithi092 ceph-mon[33665]: osdmap e77: 8 total, 8 up, 8 in 2024-05-21T12:35:01.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:01 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:01.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:01 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:01 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:01 smithi045 ceph-mon[30165]: osdmap e77: 8 total, 8 up, 8 in 2024-05-21T12:35:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:01 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:01 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:02 smithi072 ceph-mon[33912]: pgmap v594: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 42 KiB/s wr, 11 op/s 2024-05-21T12:35:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:02 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:02 smithi072 ceph-mon[33912]: osdmap e78: 8 total, 8 up, 8 in 2024-05-21T12:35:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:02 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:02 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:02.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:02 smithi092 ceph-mon[33665]: pgmap v594: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 42 KiB/s wr, 11 op/s 2024-05-21T12:35:02.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:02 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:02.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:02 smithi092 ceph-mon[33665]: osdmap e78: 8 total, 8 up, 8 in 2024-05-21T12:35:02.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:02 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:02 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:02 smithi045 ceph-mon[30165]: pgmap v594: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 11 KiB/s rd, 42 KiB/s wr, 11 op/s 2024-05-21T12:35:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:02 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:02 smithi045 ceph-mon[30165]: osdmap e78: 8 total, 8 up, 8 in 2024-05-21T12:35:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:02 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:02 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:03 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:03 smithi072 ceph-mon[33912]: osdmap e79: 8 total, 8 up, 8 in 2024-05-21T12:35:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:03 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:03 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:03.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:03 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:03.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:03 smithi092 ceph-mon[33665]: osdmap e79: 8 total, 8 up, 8 in 2024-05-21T12:35:03.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:03 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:03.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:03 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:03.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:03 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:03.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:03 smithi045 ceph-mon[30165]: osdmap e79: 8 total, 8 up, 8 in 2024-05-21T12:35:03.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:03 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:03.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:03 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:04 smithi072 ceph-mon[33912]: pgmap v597: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 44 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-05-21T12:35:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:04 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:04 smithi072 ceph-mon[33912]: osdmap e80: 8 total, 8 up, 8 in 2024-05-21T12:35:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:04 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:04.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:04 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:04 smithi092 ceph-mon[33665]: pgmap v597: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 44 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-05-21T12:35:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:04 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:04 smithi092 ceph-mon[33665]: osdmap e80: 8 total, 8 up, 8 in 2024-05-21T12:35:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:04 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:04 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:04 smithi045 ceph-mon[30165]: pgmap v597: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 44 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-05-21T12:35:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:04 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:04 smithi045 ceph-mon[30165]: osdmap e80: 8 total, 8 up, 8 in 2024-05-21T12:35:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:04 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:04 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:04.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:35:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n1 2024-05-21T12:35:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n10 2024-05-21T12:35:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n11 2024-05-21T12:35:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n12 2024-05-21T12:35:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n13 2024-05-21T12:35:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n14 2024-05-21T12:35:04.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n15 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n16 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n17 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n18 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme2n19 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme2n2 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme2n20 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme2n3 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.2k 0.0k 0.0k 12.0k 0.0k nvme2n4 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme2n5 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme2n6 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n7 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n8 2024-05-21T12:35:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n9 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.0k 0.0k 0.0k 60.0k 0.0k nvme5n1 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n10 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n11 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n12 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n13 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n14 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n15 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n16 2024-05-21T12:35:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n17 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n18 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme5n19 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme5n2 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.6k 0.0k 0.0k 56.0k 0.0k nvme5n20 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme5n3 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme5n4 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme5n5 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme5n6 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme5n7 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n8 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme8n1 2024-05-21T12:35:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n10 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n11 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n12 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n13 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n14 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n15 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n16 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n17 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.0k 0.0k 0.0k 60.0k 0.0k 0.0k nvme8n18 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme8n19 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.2k 0.0k 0.0k 52.0k 0.0k 0.0k nvme8n2 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme8n20 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.4k 0.0k 0.0k 44.0k 0.0k nvme8n3 2024-05-21T12:35:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme8n4 2024-05-21T12:35:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme8n5 2024-05-21T12:35:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme8n6 2024-05-21T12:35:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n7 2024-05-21T12:35:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:35:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n9 2024-05-21T12:35:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:05.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:05 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:05.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:05 smithi092 ceph-mon[33665]: osdmap e81: 8 total, 8 up, 8 in 2024-05-21T12:35:05.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:05 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:05.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:05 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:05 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:05 smithi045 ceph-mon[30165]: osdmap e81: 8 total, 8 up, 8 in 2024-05-21T12:35:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:05 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:05 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:05.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:05 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:05.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:05 smithi072 ceph-mon[33912]: osdmap e81: 8 total, 8 up, 8 in 2024-05-21T12:35:05.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:05 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:05.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:05 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:06 smithi092 ceph-mon[33665]: pgmap v600: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-05-21T12:35:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:06 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:06 smithi092 ceph-mon[33665]: osdmap e82: 8 total, 8 up, 8 in 2024-05-21T12:35:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:06 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:06 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:06 smithi045 ceph-mon[30165]: pgmap v600: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-05-21T12:35:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:06 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:06 smithi045 ceph-mon[30165]: osdmap e82: 8 total, 8 up, 8 in 2024-05-21T12:35:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:06 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:06.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:06 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:06 smithi072 ceph-mon[33912]: pgmap v600: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-05-21T12:35:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:06 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:06 smithi072 ceph-mon[33912]: osdmap e82: 8 total, 8 up, 8 in 2024-05-21T12:35:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:06 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:06.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:06 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:07.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:07 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:07.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:07 smithi092 ceph-mon[33665]: osdmap e83: 8 total, 8 up, 8 in 2024-05-21T12:35:07.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:07 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:07.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:07 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:07 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:07 smithi045 ceph-mon[30165]: osdmap e83: 8 total, 8 up, 8 in 2024-05-21T12:35:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:07 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:07 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:07.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:07 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:07.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:07 smithi072 ceph-mon[33912]: osdmap e83: 8 total, 8 up, 8 in 2024-05-21T12:35:07.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:07 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:07.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:07 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:08.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:08 smithi092 ceph-mon[33665]: pgmap v603: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-05-21T12:35:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:08 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:08 smithi092 ceph-mon[33665]: osdmap e84: 8 total, 8 up, 8 in 2024-05-21T12:35:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:08 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:08 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:08 smithi045 ceph-mon[30165]: pgmap v603: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-05-21T12:35:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:08 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:08.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:08 smithi045 ceph-mon[30165]: osdmap e84: 8 total, 8 up, 8 in 2024-05-21T12:35:08.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:08 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:08.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:08 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:08.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:08 smithi072 ceph-mon[33912]: pgmap v603: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 47 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-05-21T12:35:08.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:08 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:08.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:08 smithi072 ceph-mon[33912]: osdmap e84: 8 total, 8 up, 8 in 2024-05-21T12:35:08.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:08 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:08.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:08 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:09.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:09 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:09.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:09 smithi092 ceph-mon[33665]: osdmap e85: 8 total, 8 up, 8 in 2024-05-21T12:35:09.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:09 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:09.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:09 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:09 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:09 smithi045 ceph-mon[30165]: osdmap e85: 8 total, 8 up, 8 in 2024-05-21T12:35:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:09 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:09 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:09.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:09 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:09.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:09 smithi072 ceph-mon[33912]: osdmap e85: 8 total, 8 up, 8 in 2024-05-21T12:35:09.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:09 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:09.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:09 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:10.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:10 smithi092 ceph-mon[33665]: pgmap v606: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-05-21T12:35:10.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:10 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:10.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:10 smithi092 ceph-mon[33665]: osdmap e86: 8 total, 8 up, 8 in 2024-05-21T12:35:10.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:10 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:10.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:10 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:10 smithi045 ceph-mon[30165]: pgmap v606: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-05-21T12:35:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:10 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:10 smithi045 ceph-mon[30165]: osdmap e86: 8 total, 8 up, 8 in 2024-05-21T12:35:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:10 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:10 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:10 smithi072 ceph-mon[33912]: pgmap v606: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 31 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-05-21T12:35:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:10 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:10 smithi072 ceph-mon[33912]: osdmap e86: 8 total, 8 up, 8 in 2024-05-21T12:35:10.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:10 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:10.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:10 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:11.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:11 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:11.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:11 smithi092 ceph-mon[33665]: osdmap e87: 8 total, 8 up, 8 in 2024-05-21T12:35:11.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:11 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:11.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:11 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:11 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:11 smithi045 ceph-mon[30165]: osdmap e87: 8 total, 8 up, 8 in 2024-05-21T12:35:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:11 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:11 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:11.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:11 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:11.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:11 smithi072 ceph-mon[33912]: osdmap e87: 8 total, 8 up, 8 in 2024-05-21T12:35:11.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:11 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:11.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:11 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:12.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:12 smithi092 ceph-mon[33665]: pgmap v609: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-21T12:35:12.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:12 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:12.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:12 smithi092 ceph-mon[33665]: osdmap e88: 8 total, 8 up, 8 in 2024-05-21T12:35:12.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:12 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:12.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:12 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:12 smithi045 ceph-mon[30165]: pgmap v609: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-21T12:35:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:12 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:12 smithi045 ceph-mon[30165]: osdmap e88: 8 total, 8 up, 8 in 2024-05-21T12:35:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:12 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:12 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:12 smithi072 ceph-mon[33912]: pgmap v609: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 26 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-21T12:35:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:12 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:12 smithi072 ceph-mon[33912]: osdmap e88: 8 total, 8 up, 8 in 2024-05-21T12:35:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:12 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:12.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:12 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:13.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:13 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:13 smithi092 ceph-mon[33665]: osdmap e89: 8 total, 8 up, 8 in 2024-05-21T12:35:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:13 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:13 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:13 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:13 smithi045 ceph-mon[30165]: osdmap e89: 8 total, 8 up, 8 in 2024-05-21T12:35:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:13 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:13 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:13 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:13 smithi072 ceph-mon[33912]: osdmap e89: 8 total, 8 up, 8 in 2024-05-21T12:35:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:13 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:13 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:14.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:14 smithi092 ceph-mon[33665]: pgmap v612: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-05-21T12:35:14.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:14 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:14.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:14 smithi092 ceph-mon[33665]: osdmap e90: 8 total, 8 up, 8 in 2024-05-21T12:35:14.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:14 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:14.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:14 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:14 smithi045 ceph-mon[30165]: pgmap v612: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-05-21T12:35:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:14 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:14 smithi045 ceph-mon[30165]: osdmap e90: 8 total, 8 up, 8 in 2024-05-21T12:35:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:14 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:14 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:14 smithi072 ceph-mon[33912]: pgmap v612: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 97 KiB/s rd, 55 KiB/s wr, 32 op/s 2024-05-21T12:35:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:14 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:14 smithi072 ceph-mon[33912]: osdmap e90: 8 total, 8 up, 8 in 2024-05-21T12:35:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:14 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:14 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:14.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme2n1 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme2n10 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme2n11 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme2n12 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme2n13 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n14 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n15 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme2n16 2024-05-21T12:35:14.875 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.2k 0.0k 0.0k 12.0k 0.0k nvme2n17 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n18 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme2n19 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme2n2 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme2n20 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme2n3 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.0k 0.0k 0.0k 60.0k 0.0k nvme2n4 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme2n5 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n6 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme2n7 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme2n8 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.6k 0.0k 0.0k 56.0k 0.0k nvme2n9 2024-05-21T12:35:14.876 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme5n1 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.4k 0.0k 0.0k 4.0k 0.0k nvme5n10 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme5n11 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme5n12 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme5n13 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.4k 0.0k 0.0k 44.0k 0.0k 0.0k nvme5n14 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.4k 0.0k 0.0k 44.0k 0.0k 0.0k nvme5n15 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme5n16 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme5n17 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme5n18 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme5n19 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.4k 0.0k 0.0k 4.0k 0.0k nvme5n2 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme5n20 2024-05-21T12:35:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme5n3 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme5n4 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.4k 0.0k 0.0k 44.0k 0.0k 0.0k nvme5n5 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n6 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n7 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme5n8 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme5n9 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.0k 0.0k 0.0k 60.0k 0.0k nvme8n1 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme8n10 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme8n11 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme8n12 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme8n13 2024-05-21T12:35:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme8n14 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme8n15 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme8n16 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme8n17 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme8n18 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme8n19 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme8n2 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme8n20 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.8k 0.0k 0.0k 48.0k 0.0k 0.0k nvme8n3 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.4k 0.0k 0.0k 44.0k 0.0k 0.0k nvme8n4 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme8n5 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n6 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme8n7 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme8n8 2024-05-21T12:35:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme8n9 2024-05-21T12:35:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:15.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:15 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:15.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:15 smithi092 ceph-mon[33665]: osdmap e91: 8 total, 8 up, 8 in 2024-05-21T12:35:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:15 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:15 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:15 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:15 smithi045 ceph-mon[30165]: osdmap e91: 8 total, 8 up, 8 in 2024-05-21T12:35:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:15 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:15 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:15.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:15 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:15.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:15 smithi072 ceph-mon[33912]: osdmap e91: 8 total, 8 up, 8 in 2024-05-21T12:35:15.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:15 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:15.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:15 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:16.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:16 smithi092 ceph-mon[33665]: pgmap v615: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 115 KiB/s rd, 101 KiB/s wr, 35 op/s 2024-05-21T12:35:16.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:16 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:16.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:16 smithi092 ceph-mon[33665]: osdmap e92: 8 total, 8 up, 8 in 2024-05-21T12:35:16.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:16 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:16.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:16 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:16 smithi045 ceph-mon[30165]: pgmap v615: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 115 KiB/s rd, 101 KiB/s wr, 35 op/s 2024-05-21T12:35:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:16 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:16 smithi045 ceph-mon[30165]: osdmap e92: 8 total, 8 up, 8 in 2024-05-21T12:35:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:16 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:16 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:16 smithi072 ceph-mon[33912]: pgmap v615: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 115 KiB/s rd, 101 KiB/s wr, 35 op/s 2024-05-21T12:35:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:16 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:16 smithi072 ceph-mon[33912]: osdmap e92: 8 total, 8 up, 8 in 2024-05-21T12:35:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:16 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:16.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:16 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:17.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:17 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:17.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:17 smithi092 ceph-mon[33665]: osdmap e93: 8 total, 8 up, 8 in 2024-05-21T12:35:17.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:17 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:17.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:17 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:17 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:17 smithi045 ceph-mon[30165]: osdmap e93: 8 total, 8 up, 8 in 2024-05-21T12:35:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:17 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:17 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:17 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:17 smithi072 ceph-mon[33912]: osdmap e93: 8 total, 8 up, 8 in 2024-05-21T12:35:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:17 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:17 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:18.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:18 smithi092 ceph-mon[33665]: pgmap v618: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 115 KiB/s wr, 27 op/s 2024-05-21T12:35:18.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:18 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:18.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:18 smithi092 ceph-mon[33665]: osdmap e94: 8 total, 8 up, 8 in 2024-05-21T12:35:18.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:18 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:18.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:18 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:18.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:18 smithi072 ceph-mon[33912]: pgmap v618: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 115 KiB/s wr, 27 op/s 2024-05-21T12:35:18.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:18 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:18.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:18 smithi072 ceph-mon[33912]: osdmap e94: 8 total, 8 up, 8 in 2024-05-21T12:35:18.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:18 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:18.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:18 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:18.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:18 smithi045 ceph-mon[30165]: pgmap v618: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 83 KiB/s rd, 115 KiB/s wr, 27 op/s 2024-05-21T12:35:18.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:18 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:18.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:18 smithi045 ceph-mon[30165]: osdmap e94: 8 total, 8 up, 8 in 2024-05-21T12:35:18.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:18 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:18.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:18 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:19.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:19 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:19.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:19 smithi092 ceph-mon[33665]: osdmap e95: 8 total, 8 up, 8 in 2024-05-21T12:35:19.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:19 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:19.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:19 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:19 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:19 smithi045 ceph-mon[30165]: osdmap e95: 8 total, 8 up, 8 in 2024-05-21T12:35:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:19 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:19 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:19.586 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:19 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:19.587 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:19 smithi072 ceph-mon[33912]: osdmap e95: 8 total, 8 up, 8 in 2024-05-21T12:35:19.587 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:19 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:19.587 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:19 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:20 smithi045 ceph-mon[30165]: pgmap v621: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 68 KiB/s rd, 83 KiB/s wr, 25 op/s 2024-05-21T12:35:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:20 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:20 smithi045 ceph-mon[30165]: osdmap e96: 8 total, 8 up, 8 in 2024-05-21T12:35:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:20 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:20.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:20 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:20.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:20 smithi072 ceph-mon[33912]: pgmap v621: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 68 KiB/s rd, 83 KiB/s wr, 25 op/s 2024-05-21T12:35:20.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:20 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:20.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:20 smithi072 ceph-mon[33912]: osdmap e96: 8 total, 8 up, 8 in 2024-05-21T12:35:20.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:20 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:20.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:20 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:20.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:20 smithi092 ceph-mon[33665]: pgmap v621: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 68 KiB/s rd, 83 KiB/s wr, 25 op/s 2024-05-21T12:35:20.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:20 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:20.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:20 smithi092 ceph-mon[33665]: osdmap e96: 8 total, 8 up, 8 in 2024-05-21T12:35:20.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:20 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:20.670 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:20 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:21 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:21 smithi045 ceph-mon[30165]: osdmap e97: 8 total, 8 up, 8 in 2024-05-21T12:35:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:21 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:21 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:21.626 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:21 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:21.626 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:21 smithi092 ceph-mon[33665]: osdmap e97: 8 total, 8 up, 8 in 2024-05-21T12:35:21.626 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:21 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:21.626 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:21 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:21 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:21 smithi072 ceph-mon[33912]: osdmap e97: 8 total, 8 up, 8 in 2024-05-21T12:35:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:21 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:21 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:22 smithi045 ceph-mon[30165]: pgmap v624: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 34 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-21T12:35:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:22 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:22 smithi045 ceph-mon[30165]: osdmap e98: 8 total, 8 up, 8 in 2024-05-21T12:35:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:22 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:22.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:22 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:22 smithi072 ceph-mon[33912]: pgmap v624: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 34 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-21T12:35:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:22 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:22.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:22 smithi072 ceph-mon[33912]: osdmap e98: 8 total, 8 up, 8 in 2024-05-21T12:35:22.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:22 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:22.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:22 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:22 smithi092 ceph-mon[33665]: pgmap v624: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 34 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-21T12:35:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:22 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:22 smithi092 ceph-mon[33665]: osdmap e98: 8 total, 8 up, 8 in 2024-05-21T12:35:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:22 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/572653629' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:22 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"} : dispatch 2024-05-21T12:35:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:23 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:23 smithi045 ceph-mon[30165]: osdmap e99: 8 total, 8 up, 8 in 2024-05-21T12:35:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:23 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:23 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:23.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:23 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:23.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:23 smithi072 ceph-mon[33912]: osdmap e99: 8 total, 8 up, 8 in 2024-05-21T12:35:23.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:23 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:23.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:23 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:23.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:23 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.72:0/951708254"}]': finished 2024-05-21T12:35:23.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:23 smithi092 ceph-mon[33665]: osdmap e99: 8 total, 8 up, 8 in 2024-05-21T12:35:23.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:23 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/4095921432' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:23.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:23 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"} : dispatch 2024-05-21T12:35:24.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:24 smithi045 ceph-mon[30165]: pgmap v627: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 90 KiB/s wr, 33 op/s 2024-05-21T12:35:24.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:24 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:24.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:24 smithi045 ceph-mon[30165]: osdmap e100: 8 total, 8 up, 8 in 2024-05-21T12:35:24.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:24 smithi072 ceph-mon[33912]: pgmap v627: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 90 KiB/s wr, 33 op/s 2024-05-21T12:35:24.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:24 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:24.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:24 smithi072 ceph-mon[33912]: osdmap e100: 8 total, 8 up, 8 in 2024-05-21T12:35:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:24 smithi092 ceph-mon[33665]: pgmap v627: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 100 KiB/s rd, 90 KiB/s wr, 33 op/s 2024-05-21T12:35:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:24 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.c' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/3363088059"}]': finished 2024-05-21T12:35:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:24 smithi092 ceph-mon[33665]: osdmap e100: 8 total, 8 up, 8 in 2024-05-21T12:35:24.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:35:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 19.6k 21.2k 0.0k 196.0k 212.0k 0.0k nvme2n1 2024-05-21T12:35:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 19.2k 31.2k 0.0k 192.0k 312.0k 0.0k nvme2n10 2024-05-21T12:35:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 32.8k 12.4k 0.0k 328.0k 124.0k 0.0k nvme2n11 2024-05-21T12:35:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 28.0k 20.4k 0.0k 280.0k 204.0k 0.0k nvme2n12 2024-05-21T12:35:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 27.2k 19.6k 0.0k 272.0k 196.0k 0.0k nvme2n13 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 17.2k 25.2k 0.0k 172.0k 252.0k 0.0k nvme2n14 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 30.8k 12.8k 0.0k 308.0k 128.0k 0.0k nvme2n15 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 6.0k 44.8k 0.0k 60.0k 448.0k 0.0k nvme2n16 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 10.4k 20.4k 0.0k 104.0k 204.0k 0.0k nvme2n17 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 17.2k 27.6k 0.0k 172.0k 276.0k 0.0k nvme2n18 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 38.8k 25.2k 0.0k 388.0k 252.0k 0.0k nvme2n19 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 22.0k 27.2k 0.0k 220.0k 272.0k 0.0k nvme2n2 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 23.6k 22.0k 0.0k 236.0k 220.0k 0.0k nvme2n20 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 30.4k 17.2k 0.0k 304.0k 172.0k 0.0k nvme2n3 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 21.6k 27.2k 0.0k 216.0k 272.0k 0.0k nvme2n4 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.70 11.6k 32.4k 0.0k 116.0k 324.0k 0.0k nvme2n5 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 18.0k 30.4k 0.0k 180.0k 304.0k 0.0k nvme2n6 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 14.4k 34.8k 0.0k 144.0k 348.0k 0.0k nvme2n7 2024-05-21T12:35:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 15.6k 25.2k 0.0k 156.0k 252.0k 0.0k nvme2n8 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 17.6k 18.8k 0.0k 176.0k 188.0k 0.0k nvme2n9 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 21.2k 23.6k 0.0k 212.0k 236.0k 0.0k nvme5n1 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 18.0k 25.2k 0.0k 180.0k 252.0k 0.0k nvme5n10 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 19.6k 14.0k 0.0k 196.0k 140.0k 0.0k nvme5n11 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 24.0k 24.8k 0.0k 240.0k 248.0k 0.0k nvme5n12 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 21.6k 20.0k 0.0k 216.0k 200.0k 0.0k nvme5n13 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 23.6k 12.8k 0.0k 236.0k 128.0k 0.0k nvme5n14 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 15.2k 30.0k 0.0k 152.0k 300.0k 0.0k nvme5n15 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 8.4k 43.2k 0.0k 84.0k 432.0k 0.0k nvme5n16 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 20.8k 27.2k 0.0k 208.0k 272.0k 0.0k nvme5n17 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 31.2k 14.8k 0.0k 312.0k 148.0k 0.0k nvme5n18 2024-05-21T12:35:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 23.6k 26.8k 0.0k 236.0k 268.0k 0.0k nvme5n19 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 19.2k 20.4k 0.0k 192.0k 204.0k 0.0k nvme5n2 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 11.6k 27.6k 0.0k 116.0k 276.0k 0.0k nvme5n20 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 11.6k 41.2k 0.0k 116.0k 412.0k 0.0k nvme5n3 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 26.0k 19.2k 0.0k 260.0k 192.0k 0.0k nvme5n4 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.70 13.2k 22.0k 0.0k 132.0k 220.0k 0.0k nvme5n5 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 17.6k 27.2k 0.0k 176.0k 272.0k 0.0k nvme5n6 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 14.0k 36.8k 0.0k 140.0k 368.0k 0.0k nvme5n7 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 17.2k 28.4k 0.0k 172.0k 284.0k 0.0k nvme5n8 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 28.0k 18.4k 0.0k 280.0k 184.0k 0.0k nvme5n9 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 12.4k 28.4k 0.0k 124.0k 284.0k 0.0k nvme8n1 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 7.2k 30.0k 0.0k 72.0k 300.0k 0.0k nvme8n10 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 16.0k 30.8k 0.0k 160.0k 308.0k 0.0k nvme8n11 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 17.6k 36.8k 0.0k 176.0k 368.0k 0.0k nvme8n12 2024-05-21T12:35:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 34.8k 15.6k 0.0k 348.0k 156.0k 0.0k nvme8n13 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 31.2k 25.2k 0.0k 312.0k 252.0k 0.0k nvme8n14 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 24.4k 21.6k 0.0k 244.0k 216.0k 0.0k nvme8n15 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 13.6k 35.6k 0.0k 136.0k 356.0k 0.0k nvme8n16 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 26.0k 18.8k 0.0k 260.0k 188.0k 0.0k nvme8n17 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 28.4k 14.8k 0.0k 284.0k 148.0k 0.0k nvme8n18 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 30.0k 25.2k 0.0k 300.0k 252.0k 0.0k nvme8n19 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 21.2k 20.8k 0.0k 212.0k 208.0k 0.0k nvme8n2 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 14.4k 37.2k 0.0k 144.0k 372.0k 0.0k nvme8n20 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 28.4k 20.4k 0.0k 284.0k 204.0k 0.0k nvme8n3 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 26.8k 25.6k 0.0k 268.0k 256.0k 0.0k nvme8n4 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.70 24.8k 13.6k 0.0k 248.0k 136.0k 0.0k nvme8n5 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 14.0k 33.2k 0.0k 140.0k 332.0k 0.0k nvme8n6 2024-05-21T12:35:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 32.4k 11.2k 0.0k 324.0k 112.0k 0.0k nvme8n7 2024-05-21T12:35:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 20.8k 24.0k 0.0k 208.0k 240.0k 0.0k nvme8n8 2024-05-21T12:35:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 1.80 27.2k 20.8k 0.0k 272.0k 208.0k 0.0k nvme8n9 2024-05-21T12:35:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:26.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:26 smithi045 ceph-mon[30165]: pgmap v629: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 810 KiB/s rd, 912 KiB/s wr, 93 op/s 2024-05-21T12:35:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:26 smithi072 ceph-mon[33912]: pgmap v629: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 810 KiB/s rd, 912 KiB/s wr, 93 op/s 2024-05-21T12:35:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:26 smithi092 ceph-mon[33665]: pgmap v629: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 810 KiB/s rd, 912 KiB/s wr, 93 op/s 2024-05-21T12:35:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:28 smithi045 ceph-mon[30165]: pgmap v630: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-05-21T12:35:28.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:28 smithi072 ceph-mon[33912]: pgmap v630: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-05-21T12:35:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:28 smithi092 ceph-mon[33665]: pgmap v630: 33 pgs: 33 active+clean; 1023 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2024-05-21T12:35:30.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:30 smithi045 ceph-mon[30165]: pgmap v631: 33 pgs: 33 active+clean; 1024 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 303 op/s 2024-05-21T12:35:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:30 smithi072 ceph-mon[33912]: pgmap v631: 33 pgs: 33 active+clean; 1024 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 303 op/s 2024-05-21T12:35:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:30 smithi092 ceph-mon[33665]: pgmap v631: 33 pgs: 33 active+clean; 1024 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 303 op/s 2024-05-21T12:35:32.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:32 smithi045 ceph-mon[30165]: pgmap v632: 33 pgs: 33 active+clean; 1024 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-05-21T12:35:32.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:32 smithi072 ceph-mon[33912]: pgmap v632: 33 pgs: 33 active+clean; 1024 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-05-21T12:35:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:32 smithi092 ceph-mon[33665]: pgmap v632: 33 pgs: 33 active+clean; 1024 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-05-21T12:35:34.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:34 smithi045 ceph-mon[30165]: pgmap v633: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 538 op/s 2024-05-21T12:35:34.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:34 smithi072 ceph-mon[33912]: pgmap v633: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 538 op/s 2024-05-21T12:35:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:34 smithi092 ceph-mon[33665]: pgmap v633: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 538 op/s 2024-05-21T12:35:34.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 149.2k 131.2k 0.0k 1.5M 1.3M 0.0k nvme2n1 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 157.6k 107.6k 0.0k 1.5M 1.1M 0.0k nvme2n10 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 112.0k 160.4k 0.0k 1.1M 1.6M 0.0k nvme2n11 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 142.0k 144.8k 0.0k 1.4M 1.4M 0.0k nvme2n12 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 149.6k 119.2k 0.0k 1.5M 1.2M 0.0k nvme2n13 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 135.2k 157.2k 0.0k 1.3M 1.5M 0.0k nvme2n14 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 145.2k 142.4k 0.0k 1.4M 1.4M 0.0k nvme2n15 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 167.6k 124.0k 0.0k 1.6M 1.2M 0.0k nvme2n16 2024-05-21T12:35:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 121.2k 180.8k 0.0k 1.2M 1.8M 0.0k nvme2n17 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 127.2k 153.2k 0.0k 1.2M 1.5M 0.0k nvme2n18 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 121.6k 163.2k 0.0k 1.2M 1.6M 0.0k nvme2n19 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 122.8k 144.4k 0.0k 1.2M 1.4M 0.0k nvme2n2 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 131.6k 120.4k 0.0k 1.3M 1.2M 0.0k nvme2n20 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 106.0k 163.2k 0.0k 1.0M 1.6M 0.0k nvme2n3 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 137.2k 160.4k 0.0k 1.3M 1.6M 0.0k nvme2n4 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 137.6k 125.6k 0.0k 1.3M 1.2M 0.0k nvme2n5 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 134.8k 158.4k 0.0k 1.3M 1.5M 0.0k nvme2n6 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 161.6k 113.6k 0.0k 1.6M 1.1M 0.0k nvme2n7 2024-05-21T12:35:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 130.0k 115.6k 0.0k 1.3M 1.1M 0.0k nvme2n8 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 158.4k 137.6k 0.0k 1.5M 1.3M 0.0k nvme2n9 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 137.2k 125.6k 0.0k 1.3M 1.2M 0.0k nvme5n1 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 133.6k 142.8k 0.0k 1.3M 1.4M 0.0k nvme5n10 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 142.0k 113.6k 0.0k 1.4M 1.1M 0.0k nvme5n11 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 168.8k 126.4k 0.0k 1.6M 1.2M 0.0k nvme5n12 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 175.6k 110.0k 0.0k 1.7M 1.1M 0.0k nvme5n13 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 160.8k 140.0k 0.0k 1.6M 1.4M 0.0k nvme5n14 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 140.4k 143.2k 0.0k 1.4M 1.4M 0.0k nvme5n15 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 128.4k 135.6k 0.0k 1.3M 1.3M 0.0k nvme5n16 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 135.6k 122.8k 0.0k 1.3M 1.2M 0.0k nvme5n17 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 132.0k 146.4k 0.0k 1.3M 1.4M 0.0k nvme5n18 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 146.4k 154.8k 0.0k 1.4M 1.5M 0.0k nvme5n19 2024-05-21T12:35:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 144.8k 125.6k 0.0k 1.4M 1.2M 0.0k nvme5n2 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 126.8k 166.0k 0.0k 1.2M 1.6M 0.0k nvme5n20 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 140.0k 142.8k 0.0k 1.4M 1.4M 0.0k nvme5n3 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 125.6k 166.8k 0.0k 1.2M 1.6M 0.0k nvme5n4 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 150.4k 117.6k 0.0k 1.5M 1.1M 0.0k nvme5n5 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 157.6k 140.0k 0.0k 1.5M 1.4M 0.0k nvme5n6 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 145.6k 120.4k 0.0k 1.4M 1.2M 0.0k nvme5n7 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 139.2k 146.4k 0.0k 1.4M 1.4M 0.0k nvme5n8 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 115.6k 138.0k 0.0k 1.1M 1.3M 0.0k nvme5n9 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 125.6k 139.2k 0.0k 1.2M 1.4M 0.0k nvme8n1 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 143.6k 99.6k 0.0k 1.4M 996.0k 0.0k nvme8n10 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 128.0k 125.2k 0.0k 1.2M 1.2M 0.0k nvme8n11 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 117.2k 168.0k 0.0k 1.1M 1.6M 0.0k nvme8n12 2024-05-21T12:35:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 140.8k 165.2k 0.0k 1.4M 1.6M 0.0k nvme8n13 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 152.0k 113.6k 0.0k 1.5M 1.1M 0.0k nvme8n14 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 124.0k 125.6k 0.0k 1.2M 1.2M 0.0k nvme8n15 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 147.6k 138.8k 0.0k 1.4M 1.4M 0.0k nvme8n16 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 125.2k 176.4k 0.0k 1.2M 1.7M 0.0k nvme8n17 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 138.8k 112.4k 0.0k 1.4M 1.1M 0.0k nvme8n18 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 144.0k 124.0k 0.0k 1.4M 1.2M 0.0k nvme8n19 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 117.6k 180.4k 0.0k 1.1M 1.8M 0.0k nvme8n2 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 152.8k 132.4k 0.0k 1.5M 1.3M 0.0k nvme8n20 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 147.6k 134.0k 0.0k 1.4M 1.3M 0.0k nvme8n3 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 143.6k 124.4k 0.0k 1.4M 1.2M 0.0k nvme8n4 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 148.8k 150.4k 0.0k 1.5M 1.5M 0.0k nvme8n5 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 140.4k 146.8k 0.0k 1.4M 1.4M 0.0k nvme8n6 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.40 137.2k 158.8k 0.0k 1.3M 1.6M 0.0k nvme8n7 2024-05-21T12:35:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 149.6k 124.0k 0.0k 1.5M 1.2M 0.0k nvme8n8 2024-05-21T12:35:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 11.30 143.2k 151.6k 0.0k 1.4M 1.5M 0.0k nvme8n9 2024-05-21T12:35:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:36.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:36 smithi045 ceph-mon[30165]: pgmap v634: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 563 op/s 2024-05-21T12:35:36.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:36 smithi072 ceph-mon[33912]: pgmap v634: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 563 op/s 2024-05-21T12:35:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:36 smithi092 ceph-mon[33665]: pgmap v634: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 563 op/s 2024-05-21T12:35:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:38 smithi045 ceph-mon[30165]: pgmap v635: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 651 op/s 2024-05-21T12:35:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:38 smithi072 ceph-mon[33912]: pgmap v635: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 651 op/s 2024-05-21T12:35:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:38 smithi092 ceph-mon[33665]: pgmap v635: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 651 op/s 2024-05-21T12:35:40.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:40 smithi045 ceph-mon[30165]: pgmap v636: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 644 op/s 2024-05-21T12:35:40.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:40 smithi072 ceph-mon[33912]: pgmap v636: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 644 op/s 2024-05-21T12:35:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:40 smithi092 ceph-mon[33665]: pgmap v636: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 644 op/s 2024-05-21T12:35:42.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:42 smithi045 ceph-mon[30165]: pgmap v637: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 660 op/s 2024-05-21T12:35:42.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:42 smithi072 ceph-mon[33912]: pgmap v637: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 660 op/s 2024-05-21T12:35:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:42 smithi092 ceph-mon[33665]: pgmap v637: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 660 op/s 2024-05-21T12:35:44.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:44 smithi045 ceph-mon[30165]: pgmap v638: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.4 MiB/s wr, 736 op/s 2024-05-21T12:35:44.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:44 smithi072 ceph-mon[33912]: pgmap v638: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.4 MiB/s wr, 736 op/s 2024-05-21T12:35:44.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:44 smithi092 ceph-mon[33665]: pgmap v638: 33 pgs: 33 active+clean; 1024 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.4 MiB/s wr, 736 op/s 2024-05-21T12:35:44.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:44.876 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 123.6k 116.4k 0.0k 1.2M 1.1M 0.0k nvme2n1 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 114.0k 119.2k 0.0k 1.1M 1.2M 0.0k nvme2n10 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 104.8k 113.2k 0.0k 1.0M 1.1M 0.0k nvme2n11 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 124.0k 106.4k 0.0k 1.2M 1.0M 0.0k nvme2n12 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 144.4k 98.4k 0.0k 1.4M 984.0k 0.0k nvme2n13 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 140.8k 116.4k 0.0k 1.4M 1.1M 0.0k nvme2n14 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 96.0k 139.6k 0.0k 960.0k 1.4M 0.0k nvme2n15 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 138.4k 139.2k 0.0k 1.4M 1.4M 0.0k nvme2n16 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 125.6k 126.4k 0.0k 1.2M 1.2M 0.0k nvme2n17 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 164.0k 99.6k 0.0k 1.6M 996.0k 0.0k nvme2n18 2024-05-21T12:35:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 124.8k 108.0k 0.0k 1.2M 1.1M 0.0k nvme2n19 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 111.6k 132.4k 0.0k 1.1M 1.3M 0.0k nvme2n2 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 135.6k 109.6k 0.0k 1.3M 1.1M 0.0k nvme2n20 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 148.0k 127.2k 0.0k 1.4M 1.2M 0.0k nvme2n3 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 131.2k 148.4k 0.0k 1.3M 1.4M 0.0k nvme2n4 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 112.8k 128.8k 0.0k 1.1M 1.3M 0.0k nvme2n5 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 109.2k 123.6k 0.0k 1.1M 1.2M 0.0k nvme2n6 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 106.0k 123.2k 0.0k 1.0M 1.2M 0.0k nvme2n7 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 116.4k 119.6k 0.0k 1.1M 1.2M 0.0k nvme2n8 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 138.4k 107.6k 0.0k 1.4M 1.1M 0.0k nvme2n9 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 134.4k 92.8k 0.0k 1.3M 928.0k 0.0k nvme5n1 2024-05-21T12:35:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 122.4k 123.2k 0.0k 1.2M 1.2M 0.0k nvme5n10 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 135.2k 108.0k 0.0k 1.3M 1.1M 0.0k nvme5n11 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 138.8k 130.0k 0.0k 1.4M 1.3M 0.0k nvme5n12 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 137.2k 99.6k 0.0k 1.3M 996.0k 0.0k nvme5n13 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 132.0k 97.6k 0.0k 1.3M 976.0k 0.0k nvme5n14 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 112.0k 139.2k 0.0k 1.1M 1.4M 0.0k nvme5n15 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 102.8k 141.2k 0.0k 1.0M 1.4M 0.0k nvme5n16 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 118.0k 126.4k 0.0k 1.2M 1.2M 0.0k nvme5n17 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 146.4k 124.0k 0.0k 1.4M 1.2M 0.0k nvme5n18 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 124.0k 136.4k 0.0k 1.2M 1.3M 0.0k nvme5n19 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 111.2k 124.8k 0.0k 1.1M 1.2M 0.0k nvme5n2 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 118.8k 146.0k 0.0k 1.2M 1.4M 0.0k nvme5n20 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 141.6k 122.4k 0.0k 1.4M 1.2M 0.0k nvme5n3 2024-05-21T12:35:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 181.2k 84.8k 0.0k 1.8M 848.0k 0.0k nvme5n4 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 101.6k 143.6k 0.0k 1016.0k 1.4M 0.0k nvme5n5 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 106.4k 129.6k 0.0k 1.0M 1.3M 0.0k nvme5n6 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 96.8k 156.8k 0.0k 968.0k 1.5M 0.0k nvme5n7 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 144.4k 124.4k 0.0k 1.4M 1.2M 0.0k nvme5n8 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 108.8k 110.8k 0.0k 1.1M 1.1M 0.0k nvme5n9 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 121.2k 139.6k 0.0k 1.2M 1.4M 0.0k nvme8n1 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 107.2k 117.2k 0.0k 1.0M 1.1M 0.0k nvme8n10 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 137.6k 114.0k 0.0k 1.3M 1.1M 0.0k nvme8n11 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 133.2k 138.8k 0.0k 1.3M 1.4M 0.0k nvme8n12 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 110.0k 128.8k 0.0k 1.1M 1.3M 0.0k nvme8n13 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 142.4k 128.8k 0.0k 1.4M 1.3M 0.0k nvme8n14 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 119.6k 162.0k 0.0k 1.2M 1.6M 0.0k nvme8n15 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 116.8k 130.0k 0.0k 1.1M 1.3M 0.0k nvme8n16 2024-05-21T12:35:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 150.0k 110.0k 0.0k 1.5M 1.1M 0.0k nvme8n17 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 125.6k 114.8k 0.0k 1.2M 1.1M 0.0k nvme8n18 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 136.4k 126.4k 0.0k 1.3M 1.2M 0.0k nvme8n19 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 151.6k 108.0k 0.0k 1.5M 1.1M 0.0k nvme8n2 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 101.2k 132.4k 0.0k 1012.0k 1.3M 0.0k nvme8n20 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 135.6k 119.2k 0.0k 1.3M 1.2M 0.0k nvme8n3 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 111.2k 135.6k 0.0k 1.1M 1.3M 0.0k nvme8n4 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 119.2k 119.6k 0.0k 1.2M 1.2M 0.0k nvme8n5 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 133.2k 105.6k 0.0k 1.3M 1.0M 0.0k nvme8n6 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 110.8k 114.4k 0.0k 1.1M 1.1M 0.0k nvme8n7 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 98.8k 116.0k 0.0k 988.0k 1.1M 0.0k nvme8n8 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 131.2k 128.4k 0.0k 1.3M 1.3M 0.0k nvme8n9 2024-05-21T12:35:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:46.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:46 smithi045 ceph-mon[30165]: pgmap v639: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 668 op/s 2024-05-21T12:35:46.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:46 smithi072 ceph-mon[33912]: pgmap v639: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 668 op/s 2024-05-21T12:35:46.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:46 smithi092 ceph-mon[33665]: pgmap v639: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 668 op/s 2024-05-21T12:35:48.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:48 smithi045 ceph-mon[30165]: pgmap v640: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.3 MiB/s wr, 683 op/s 2024-05-21T12:35:48.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:48 smithi072 ceph-mon[33912]: pgmap v640: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.3 MiB/s wr, 683 op/s 2024-05-21T12:35:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:48 smithi092 ceph-mon[33665]: pgmap v640: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.3 MiB/s wr, 683 op/s 2024-05-21T12:35:50.553 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:50 smithi092 ceph-mon[33665]: pgmap v641: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.7 MiB/s wr, 638 op/s 2024-05-21T12:35:50.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:50 smithi045 ceph-mon[30165]: pgmap v641: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.7 MiB/s wr, 638 op/s 2024-05-21T12:35:50.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:50 smithi072 ceph-mon[33912]: pgmap v641: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.7 MiB/s wr, 638 op/s 2024-05-21T12:35:52.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:52 smithi045 ceph-mon[30165]: pgmap v642: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 652 op/s 2024-05-21T12:35:52.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:52 smithi072 ceph-mon[33912]: pgmap v642: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 652 op/s 2024-05-21T12:35:52.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:52 smithi092 ceph-mon[33665]: pgmap v642: 33 pgs: 33 active+clean; 1024 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 652 op/s 2024-05-21T12:35:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:54 smithi045 ceph-mon[30165]: pgmap v643: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 725 op/s 2024-05-21T12:35:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:54 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:35:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:54 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:35:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:54 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:35:54.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:54 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:35:54.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:54 smithi072 ceph-mon[33912]: pgmap v643: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 725 op/s 2024-05-21T12:35:54.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:54 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:35:54.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:54 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:35:54.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:54 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:35:54.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:54 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:35:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:54 smithi092 ceph-mon[33665]: pgmap v643: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 725 op/s 2024-05-21T12:35:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:54 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:35:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:54 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:35:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:54 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:35:54.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:54 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:35:54.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:35:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 112.8k 141.2k 0.0k 1.1M 1.4M 0.0k nvme2n1 2024-05-21T12:35:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 108.0k 126.4k 0.0k 1.1M 1.2M 0.0k nvme2n10 2024-05-21T12:35:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 128.4k 100.8k 0.0k 1.3M 1008.0k 0.0k nvme2n11 2024-05-21T12:35:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 140.4k 94.0k 0.0k 1.4M 940.0k 0.0k nvme2n12 2024-05-21T12:35:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 118.8k 124.8k 0.0k 1.2M 1.2M 0.0k nvme2n13 2024-05-21T12:35:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 125.6k 140.0k 0.0k 1.2M 1.4M 0.0k nvme2n14 2024-05-21T12:35:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 124.0k 107.6k 0.0k 1.2M 1.1M 0.0k nvme2n15 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 96.0k 162.8k 0.0k 960.0k 1.6M 0.0k nvme2n16 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 117.6k 105.6k 0.0k 1.1M 1.0M 0.0k nvme2n17 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 117.6k 95.6k 0.0k 1.1M 956.0k 0.0k nvme2n18 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 128.8k 104.0k 0.0k 1.3M 1.0M 0.0k nvme2n19 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 108.0k 90.4k 0.0k 1.1M 904.0k 0.0k nvme2n2 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 88.4k 116.4k 0.0k 884.0k 1.1M 0.0k nvme2n20 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 108.4k 123.2k 0.0k 1.1M 1.2M 0.0k nvme2n3 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 117.2k 94.0k 0.0k 1.1M 940.0k 0.0k nvme2n4 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 122.0k 95.6k 0.0k 1.2M 956.0k 0.0k nvme2n5 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 101.6k 112.0k 0.0k 1016.0k 1.1M 0.0k nvme2n6 2024-05-21T12:35:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 100.4k 115.6k 0.0k 1004.0k 1.1M 0.0k nvme2n7 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 105.6k 117.6k 0.0k 1.0M 1.1M 0.0k nvme2n8 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 105.2k 125.6k 0.0k 1.0M 1.2M 0.0k nvme2n9 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 97.6k 132.0k 0.0k 976.0k 1.3M 0.0k nvme5n1 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 101.6k 134.0k 0.0k 1016.0k 1.3M 0.0k nvme5n10 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 113.2k 121.2k 0.0k 1.1M 1.2M 0.0k nvme5n11 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 137.6k 104.8k 0.0k 1.3M 1.0M 0.0k nvme5n12 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 106.4k 124.4k 0.0k 1.0M 1.2M 0.0k nvme5n13 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 120.4k 98.4k 0.0k 1.2M 984.0k 0.0k nvme5n14 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 100.0k 120.4k 0.0k 1000.0k 1.2M 0.0k nvme5n15 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 131.6k 98.4k 0.0k 1.3M 984.0k 0.0k nvme5n16 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 105.6k 130.8k 0.0k 1.0M 1.3M 0.0k nvme5n17 2024-05-21T12:35:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 126.8k 113.2k 0.0k 1.2M 1.1M 0.0k nvme5n18 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 120.8k 98.4k 0.0k 1.2M 984.0k 0.0k nvme5n19 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 136.4k 117.2k 0.0k 1.3M 1.1M 0.0k nvme5n2 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 123.2k 90.8k 0.0k 1.2M 908.0k 0.0k nvme5n20 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 120.0k 130.4k 0.0k 1.2M 1.3M 0.0k nvme5n3 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 139.2k 96.8k 0.0k 1.4M 968.0k 0.0k nvme5n4 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 110.0k 111.2k 0.0k 1.1M 1.1M 0.0k nvme5n5 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 108.8k 144.8k 0.0k 1.1M 1.4M 0.0k nvme5n6 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 104.8k 139.6k 0.0k 1.0M 1.4M 0.0k nvme5n7 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 128.4k 114.4k 0.0k 1.3M 1.1M 0.0k nvme5n8 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 122.8k 103.6k 0.0k 1.2M 1.0M 0.0k nvme5n9 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 103.6k 127.6k 0.0k 1.0M 1.2M 0.0k nvme8n1 2024-05-21T12:35:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 116.0k 111.2k 0.0k 1.1M 1.1M 0.0k nvme8n10 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 80.8k 128.4k 0.0k 808.0k 1.3M 0.0k nvme8n11 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 98.0k 136.4k 0.0k 980.0k 1.3M 0.0k nvme8n12 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 103.2k 118.4k 0.0k 1.0M 1.2M 0.0k nvme8n13 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 109.2k 94.8k 0.0k 1.1M 948.0k 0.0k nvme8n14 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 86.4k 138.4k 0.0k 864.0k 1.4M 0.0k nvme8n15 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 128.8k 124.4k 0.0k 1.3M 1.2M 0.0k nvme8n16 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 122.8k 104.4k 0.0k 1.2M 1.0M 0.0k nvme8n17 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 110.8k 125.2k 0.0k 1.1M 1.2M 0.0k nvme8n18 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 104.0k 132.0k 0.0k 1.0M 1.3M 0.0k nvme8n19 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 124.8k 90.4k 0.0k 1.2M 904.0k 0.0k nvme8n2 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 128.8k 100.8k 0.0k 1.3M 1008.0k 0.0k nvme8n20 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 94.8k 148.0k 0.0k 948.0k 1.4M 0.0k nvme8n3 2024-05-21T12:35:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 104.0k 158.0k 0.0k 1.0M 1.5M 0.0k nvme8n4 2024-05-21T12:35:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 109.2k 105.2k 0.0k 1.1M 1.0M 0.0k nvme8n5 2024-05-21T12:35:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 134.0k 100.8k 0.0k 1.3M 1008.0k 0.0k nvme8n6 2024-05-21T12:35:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 119.2k 99.2k 0.0k 1.2M 992.0k 0.0k nvme8n7 2024-05-21T12:35:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 125.2k 111.6k 0.0k 1.2M 1.1M 0.0k nvme8n8 2024-05-21T12:35:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 126.4k 124.8k 0.0k 1.2M 1.2M 0.0k nvme8n9 2024-05-21T12:35:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:35:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:55 smithi045 ceph-mon[30165]: pgmap v644: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-05-21T12:35:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:55 smithi045 ceph-mon[30165]: pgmap v645: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 635 op/s 2024-05-21T12:35:55.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:55 smithi072 ceph-mon[33912]: pgmap v644: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-05-21T12:35:55.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:55 smithi072 ceph-mon[33912]: pgmap v645: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 635 op/s 2024-05-21T12:35:55.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:55 smithi092 ceph-mon[33665]: pgmap v644: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 629 op/s 2024-05-21T12:35:55.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:55 smithi092 ceph-mon[33665]: pgmap v645: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 635 op/s 2024-05-21T12:35:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:57 smithi045 ceph-mon[30165]: pgmap v646: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 594 op/s 2024-05-21T12:35:57.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:57 smithi072 ceph-mon[33912]: pgmap v646: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 594 op/s 2024-05-21T12:35:57.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:57 smithi092 ceph-mon[33665]: pgmap v646: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 594 op/s 2024-05-21T12:35:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:35:59 smithi045 ceph-mon[30165]: pgmap v647: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 720 op/s 2024-05-21T12:35:59.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:35:59 smithi072 ceph-mon[33912]: pgmap v647: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 720 op/s 2024-05-21T12:35:59.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:35:59 smithi092 ceph-mon[33665]: pgmap v647: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 720 op/s 2024-05-21T12:36:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:01 smithi045 ceph-mon[30165]: pgmap v648: 33 pgs: 33 active+clean; 1024 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2024-05-21T12:36:01.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:01 smithi072 ceph-mon[33912]: pgmap v648: 33 pgs: 33 active+clean; 1024 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2024-05-21T12:36:01.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:01 smithi092 ceph-mon[33665]: pgmap v648: 33 pgs: 33 active+clean; 1024 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2024-05-21T12:36:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:03 smithi045 ceph-mon[30165]: pgmap v649: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 684 op/s 2024-05-21T12:36:03.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:03 smithi072 ceph-mon[33912]: pgmap v649: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 684 op/s 2024-05-21T12:36:03.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:03 smithi092 ceph-mon[33665]: pgmap v649: 33 pgs: 33 active+clean; 1024 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 684 op/s 2024-05-21T12:36:04.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:36:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 138.4k 113.2k 0.0k 1.4M 1.1M 0.0k nvme2n1 2024-05-21T12:36:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 88.0k 153.2k 0.0k 880.0k 1.5M 0.0k nvme2n10 2024-05-21T12:36:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 92.8k 105.2k 0.0k 928.0k 1.0M 0.0k nvme2n11 2024-05-21T12:36:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 117.6k 93.2k 0.0k 1.1M 932.0k 0.0k nvme2n12 2024-05-21T12:36:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 102.0k 93.2k 0.0k 1020.0k 932.0k 0.0k nvme2n13 2024-05-21T12:36:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 77.6k 119.6k 0.0k 776.0k 1.2M 0.0k nvme2n14 2024-05-21T12:36:04.876 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 106.8k 96.0k 0.0k 1.0M 960.0k 0.0k nvme2n15 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 129.6k 65.2k 0.0k 1.3M 652.0k 0.0k nvme2n16 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 92.4k 120.4k 0.0k 924.0k 1.2M 0.0k nvme2n17 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 119.6k 88.4k 0.0k 1.2M 884.0k 0.0k nvme2n18 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 119.2k 94.4k 0.0k 1.2M 944.0k 0.0k nvme2n19 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 94.8k 114.8k 0.0k 948.0k 1.1M 0.0k nvme2n2 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 118.0k 146.0k 0.0k 1.2M 1.4M 0.0k nvme2n20 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 116.0k 105.6k 0.0k 1.1M 1.0M 0.0k nvme2n3 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 111.2k 86.0k 0.0k 1.1M 860.0k 0.0k nvme2n4 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 117.2k 80.4k 0.0k 1.1M 804.0k 0.0k nvme2n5 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 124.4k 131.6k 0.0k 1.2M 1.3M 0.0k nvme2n6 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 115.2k 152.0k 0.0k 1.1M 1.5M 0.0k nvme2n7 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 127.6k 134.4k 0.0k 1.2M 1.3M 0.0k nvme2n8 2024-05-21T12:36:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 114.8k 123.6k 0.0k 1.1M 1.2M 0.0k nvme2n9 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 118.4k 120.0k 0.0k 1.2M 1.2M 0.0k nvme5n1 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 102.4k 92.4k 0.0k 1.0M 924.0k 0.0k nvme5n10 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 94.8k 105.2k 0.0k 948.0k 1.0M 0.0k nvme5n11 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 132.8k 82.0k 0.0k 1.3M 820.0k 0.0k nvme5n12 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 103.6k 90.8k 0.0k 1.0M 908.0k 0.0k nvme5n13 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 96.4k 111.6k 0.0k 964.0k 1.1M 0.0k nvme5n14 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 110.8k 107.2k 0.0k 1.1M 1.0M 0.0k nvme5n15 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 112.4k 119.2k 0.0k 1.1M 1.2M 0.0k nvme5n16 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 149.6k 138.4k 0.0k 1.5M 1.4M 0.0k nvme5n17 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 104.4k 92.8k 0.0k 1.0M 928.0k 0.0k nvme5n18 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 97.2k 104.4k 0.0k 972.0k 1.0M 0.0k nvme5n19 2024-05-21T12:36:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 130.4k 112.8k 0.0k 1.3M 1.1M 0.0k nvme5n2 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 102.4k 94.0k 0.0k 1.0M 940.0k 0.0k nvme5n20 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 108.4k 90.8k 0.0k 1.1M 908.0k 0.0k nvme5n3 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 92.4k 110.8k 0.0k 924.0k 1.1M 0.0k nvme5n4 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 113.6k 92.8k 0.0k 1.1M 928.0k 0.0k nvme5n5 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 102.8k 94.0k 0.0k 1.0M 940.0k 0.0k nvme5n6 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 125.6k 105.2k 0.0k 1.2M 1.0M 0.0k nvme5n7 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 123.6k 96.0k 0.0k 1.2M 960.0k 0.0k nvme5n8 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 132.4k 136.8k 0.0k 1.3M 1.3M 0.0k nvme5n9 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 98.8k 98.8k 0.0k 988.0k 988.0k 0.0k nvme8n1 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 145.2k 124.4k 0.0k 1.4M 1.2M 0.0k nvme8n10 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 98.0k 123.2k 0.0k 980.0k 1.2M 0.0k nvme8n11 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 86.0k 77.2k 0.0k 860.0k 772.0k 0.0k nvme8n12 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 148.4k 88.8k 0.0k 1.4M 888.0k 0.0k nvme8n13 2024-05-21T12:36:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 142.8k 120.4k 0.0k 1.4M 1.2M 0.0k nvme8n14 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.10 100.8k 92.8k 0.0k 1008.0k 928.0k 0.0k nvme8n15 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 173.2k 134.0k 0.0k 1.7M 1.3M 0.0k nvme8n16 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 107.6k 128.0k 0.0k 1.1M 1.2M 0.0k nvme8n17 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 95.6k 107.2k 0.0k 956.0k 1.0M 0.0k nvme8n18 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 112.4k 97.6k 0.0k 1.1M 976.0k 0.0k nvme8n19 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 126.0k 96.8k 0.0k 1.2M 968.0k 0.0k nvme8n2 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 113.2k 112.0k 0.0k 1.1M 1.1M 0.0k nvme8n20 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 110.8k 115.2k 0.0k 1.1M 1.1M 0.0k nvme8n3 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 96.0k 115.6k 0.0k 960.0k 1.1M 0.0k nvme8n4 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 101.2k 112.0k 0.0k 1012.0k 1.1M 0.0k nvme8n5 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 140.0k 70.0k 0.0k 1.4M 700.0k 0.0k nvme8n6 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 80.0k 101.2k 0.0k 800.0k 1012.0k 0.0k nvme8n7 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 126.0k 122.4k 0.0k 1.2M 1.2M 0.0k nvme8n8 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 11.20 92.4k 114.4k 0.0k 924.0k 1.1M 0.0k nvme8n9 2024-05-21T12:36:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:05.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:05 smithi072 ceph-mon[33912]: pgmap v650: 33 pgs: 33 active+clean; 1.0 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 680 op/s 2024-05-21T12:36:05.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:05 smithi092 ceph-mon[33665]: pgmap v650: 33 pgs: 33 active+clean; 1.0 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 680 op/s 2024-05-21T12:36:05.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:05 smithi045 ceph-mon[30165]: pgmap v650: 33 pgs: 33 active+clean; 1.0 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 680 op/s 2024-05-21T12:36:07.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:07 smithi072 ceph-mon[33912]: pgmap v651: 33 pgs: 33 active+clean; 1.0 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 657 op/s 2024-05-21T12:36:07.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:07 smithi092 ceph-mon[33665]: pgmap v651: 33 pgs: 33 active+clean; 1.0 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 657 op/s 2024-05-21T12:36:07.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:07 smithi045 ceph-mon[30165]: pgmap v651: 33 pgs: 33 active+clean; 1.0 GiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 657 op/s 2024-05-21T12:36:08.906 INFO:tasks.nvmeof.[nvmeof.thrasher]:done waiting before reviving 2024-05-21T12:36:08.907 INFO:tasks.nvmeof.[nvmeof.thrasher]:display and verify stats: 2024-05-21T12:36:08.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:36:08.942 INFO:teuthology.orchestra.run.smithi045.stdout:× ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service - Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:36:08.942 INFO:teuthology.orchestra.run.smithi045.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:36:08.942 INFO:teuthology.orchestra.run.smithi045.stdout: Active: failed (Result: exit-code) since Tue 2024-05-21 12:34:17 UTC; 1min 51s ago 2024-05-21T12:36:08.942 INFO:teuthology.orchestra.run.smithi045.stdout: Duration: 1min 26.620s 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 72823 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 72824 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 74354 ExecStop=/bin/bash -c bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.stop (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 74426 ExecStopPost=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.poststop (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 74458 ExecStopPost=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout: Main PID: 72885 (code=exited, status=137) 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout: CPU: 1min 40.611s 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:06 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[72885]: [21-May-2024 12:34:06] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log.gz 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:16 smithi045 bash[74384]: time="2024-05-21T12:34:16Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a in 10 seconds, resorting to SIGKILL" 2024-05-21T12:36:08.943 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:16 smithi045 podman[74384]: 2024-05-21 12:34:16.532729287 +0000 UTC m=+10.136799303 container died c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, architecture=x86_64, io.ceph.component=ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, org.centos.version=9.0-21.el9, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.version=1.2.9, io.spdk.build.ceph-release=19.0.0, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, com.redhat.component=ubi9-container, io.spdk.git.repo=https://github.com/ceph/spdk.git, vcs-type=git, io.spdk.build.makeflags=, io.openshift.wants=ceph,rbd, io.buildah.version=1.29.0, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.target-arch=, io.spdk.build.pkgdep-args=--rbd, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, url=https://github.com/ceph/ceph-nvmeof, version=1.2.9, build-date=2024-05-19 17:57:19 UTC, distribution-scope=public, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.version=24.01, vendor=Red Hat, Inc., release=, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., summary=Ceph NVMe over Fabrics Gateway, io.ceph.summary=Ceph NVMe over Fabrics Gateway, name=ceph-nvmeof, io.spdk.git.branch=undefined, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.maintainer=Ceph Developers , io.ceph.nvmeof=None) 2024-05-21T12:36:08.944 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:17 smithi045 podman[74384]: 2024-05-21 12:34:17.18461411 +0000 UTC m=+10.788684123 container cleanup c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, summary=Ceph NVMe over Fabrics Gateway, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.maintainer=Ceph Developers , io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.version=1.2.9, vendor=Red Hat, Inc., io.ceph.git.branch=tags/1.2.9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.pkgdep-args=--rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.version=24.01, url=https://github.com/ceph/ceph-nvmeof, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., version=1.2.9, org.centos.version=9.0-21.el9, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, io.ceph.component=ceph-nvmeof, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, build-date=2024-05-19 17:57:19 UTC, io.openshift.wants=ceph,rbd, io.ceph.nvmeof=None, io.buildah.version=1.29.0, architecture=x86_64, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, release=, io.spdk.git.branch=undefined, io.spdk.build.ceph-release=19.0.0, com.redhat.component=ubi9-container, maintainer=Ceph Developers , name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-type=git, distribution-scope=public, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/) 2024-05-21T12:36:08.944 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:17 smithi045 bash[74384]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a 2024-05-21T12:36:08.944 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:17 smithi045 podman[74414]: 2024-05-21 12:34:17.469454249 +0000 UTC m=+0.931345326 container remove c569d1362709f3c8adad67a27c5a449584f9e427cf2b4b39e0e04a9984a335db (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.branch=undefined, com.redhat.component=ubi9-container, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.target-arch=, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, version=1.2.9, io.buildah.version=1.29.0, io.ceph.version=1.2.9, url=https://github.com/ceph/ceph-nvmeof, build-date=2024-05-19 17:57:19 UTC, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, vcs-type=git, io.ceph.component=ceph-nvmeof, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.openshift.wants=ceph,rbd, io.spdk.git.repo=https://github.com/ceph/spdk.git, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.version=24.01, release=, vendor=Red Hat, Inc., io.spdk.build.ceph-release=19.0.0, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.git.branch=tags/1.2.9, org.centos.version=9.0-21.el9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.maintainer=Ceph Developers , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.nvmeof=None, io.spdk.build.makeflags=, maintainer=Ceph Developers , name=ceph-nvmeof, distribution-scope=public, architecture=x86_64) 2024-05-21T12:36:08.944 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:17 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:36:08.944 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:17 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Failed with result 'exit-code'. 2024-05-21T12:36:08.944 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:17 smithi045 systemd[1]: Stopped Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:36:08.944 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:34:17 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Consumed 1min 40.611s CPU time. 2024-05-21T12:36:08.945 DEBUG:teuthology.orchestra.run:got remote process result: 3 2024-05-21T12:36:08.945 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:36:08.979 INFO:teuthology.orchestra.run.smithi072.stdout:× ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service - Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:36:08.979 INFO:teuthology.orchestra.run.smithi072.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:36:08.979 INFO:teuthology.orchestra.run.smithi072.stdout: Active: failed (Result: exit-code) since Tue 2024-05-21 12:34:28 UTC; 1min 40s ago 2024-05-21T12:36:08.979 INFO:teuthology.orchestra.run.smithi072.stdout: Duration: 10min 7.283s 2024-05-21T12:36:08.979 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 53097 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.979 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 53098 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.979 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56202 ExecStop=/bin/bash -c bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.stop (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56272 ExecStopPost=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.poststop (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56303 ExecStopPost=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout: Main PID: 53263 (code=exited, status=137) 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout: CPU: 10min 18.625s 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:17 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[53263]: [21-May-2024 12:34:17] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.b/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.b/nvmeof-log.gz 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:27 smithi072 bash[56230]: time="2024-05-21T12:34:27Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b in 10 seconds, resorting to SIGKILL" 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:28 smithi072 podman[56230]: 2024-05-21 12:34:28.006771742 +0000 UTC m=+10.142767636 container died a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.version=24.01, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, com.redhat.component=ubi9-container, version=1.2.9, summary=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.git.branch=undefined, io.spdk.build.target-arch=, url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.build.ceph-release=19.0.0, io.ceph.nvmeof=None, vendor=Red Hat, Inc., io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, name=ceph-nvmeof, build-date=2024-05-19 17:57:19 UTC, io.buildah.version=1.29.0, io.ceph.summary=Ceph NVMe over Fabrics Gateway, maintainer=Ceph Developers , io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., architecture=x86_64, release=, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.maintainer=Ceph Developers , org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, distribution-scope=public, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.pkgdep-args=--rbd, io.ceph.component=ceph-nvmeof, vcs-type=git, io.ceph.url=https://github.com/ceph/ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.makeflags=, io.ceph.version=1.2.9, org.centos.version=9.0-21.el9, io.ceph.git.branch=tags/1.2.9) 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:28 smithi072 podman[56230]: 2024-05-21 12:34:28.450610165 +0000 UTC m=+10.586606101 container cleanup a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.spdk.build.pkgdep-args=--rbd, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, maintainer=Ceph Developers , io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.nvmeof=None, io.openshift.wants=ceph,rbd, io.spdk.build.target-arch=, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, vcs-type=git, org.centos.version=9.0-21.el9, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.url=https://github.com/ceph/ceph-nvmeof, build-date=2024-05-19 17:57:19 UTC, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, vendor=Red Hat, Inc., description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.buildah.version=1.29.0, io.spdk.build.ceph-release=19.0.0, summary=Ceph NVMe over Fabrics Gateway, distribution-scope=public, io.spdk.git.branch=undefined, io.spdk.version=24.01, com.redhat.component=ubi9-container, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, architecture=x86_64, io.ceph.version=1.2.9, io.ceph.maintainer=Ceph Developers , io.ceph.component=ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://github.com/ceph/ceph-nvmeof, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.git.repo=https://github.com/ceph/spdk.git, release=, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, version=1.2.9) 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:28 smithi072 bash[56230]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b 2024-05-21T12:36:08.980 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:28 smithi072 podman[56258]: 2024-05-21 12:34:28.620056554 +0000 UTC m=+0.612793502 container remove a048ad7abf639054a3c32b5846054741edc3a5fc2f679284cb533c055656b243 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.ceph.git.branch=tags/1.2.9, vcs-type=git, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.makeflags=, io.spdk.version=24.01, io.ceph.maintainer=Ceph Developers , release=, version=1.2.9, io.buildah.version=1.29.0, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.nvmeof=None, io.ceph.version=1.2.9, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.build.ceph-release=19.0.0, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., summary=Ceph NVMe over Fabrics Gateway, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, com.redhat.component=ubi9-container, vendor=Red Hat, Inc., io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.git.repo=https://github.com/ceph/spdk.git, distribution-scope=public, org.centos.version=9.0-21.el9, io.spdk.build.target-arch=, architecture=x86_64, io.spdk.git.branch=undefined, maintainer=Ceph Developers , io.ceph.component=ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.summary=Ceph NVMe over Fabrics Gateway, name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.wants=ceph,rbd, url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, build-date=2024-05-19 17:57:19 UTC, io.ceph.url=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:36:08.981 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:28 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:36:08.981 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:28 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Failed with result 'exit-code'. 2024-05-21T12:36:08.981 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:28 smithi072 systemd[1]: Stopped Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:36:08.981 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:34:28 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Consumed 10min 18.625s CPU time. 2024-05-21T12:36:08.981 DEBUG:teuthology.orchestra.run:got remote process result: 3 2024-05-21T12:36:08.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:36:09.017 INFO:teuthology.orchestra.run.smithi092.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service - Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:36:09.017 INFO:teuthology.orchestra.run.smithi092.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:36:09.017 INFO:teuthology.orchestra.run.smithi092.stdout: Active: active (running) since Tue 2024-05-21 12:23:56 UTC; 12min ago 2024-05-21T12:36:09.017 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53502 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:36:09.017 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53503 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:36:09.017 INFO:teuthology.orchestra.run.smithi092.stdout: Main PID: 53679 (conmon) 2024-05-21T12:36:09.017 INFO:teuthology.orchestra.run.smithi092.stdout: Tasks: 126 (limit: 202816) 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: Memory: 1.1G 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: CPU: 12min 845ms 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: ├─libpod-payload-5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53681 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53683 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53701 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.c --gateway-address 172.21.15.92:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.92:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.c -k /etc/ceph/keyring 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─54233 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: │ └─54267 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: └─runtime 2024-05-21T12:36:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: └─53679 /usr/bin/conmon --api-version 1 -c 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d -u 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata -p /run/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: OPTIMIZED 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:48746 2024-05-21T12:36:09.019 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:34:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [2024-05-21 12:34:44.042504] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000202 seconds 2024-05-21T12:36:09.020 DEBUG:teuthology.orchestra.run.smithi137:> ceph orch ls && ceph orch ps --daemon-type nvmeof && ceph health detail && ceph -s && sudo nvme list-subsys /dev/nvme5n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n20 | grep 'live optimized' 2024-05-21T12:36:09.332 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T12:36:09.333 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 1/1 76s ago 17m smithi045=x;count:1 2024-05-21T12:36:09.333 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 77s ago 17m smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:36:09.333 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.mypool ?:4420,5500,8009 1/3 77s ago 12m smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:36:09.333 INFO:teuthology.orchestra.run.smithi137.stdout:osd 8 77s ago - 2024-05-21T12:36:09.624 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:36:09.625 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.a smithi045 *:5500,4420,8009 error 77s ago 11m - - 2024-05-21T12:36:09.625 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.b smithi072 *:5500,4420,8009 error 77s ago 11m - - 2024-05-21T12:36:09.625 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.c smithi092 *:5500,4420,8009 running (12m) 77s ago 12m 160M - 1.2.9 30dd525ace30 5e5a8ffd2ba6 2024-05-21T12:36:09.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:09 smithi072 ceph-mon[33912]: pgmap v652: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 735 op/s 2024-05-21T12:36:09.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:09 smithi092 ceph-mon[33665]: pgmap v652: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 735 op/s 2024-05-21T12:36:09.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:09 smithi045 ceph-mon[30165]: pgmap v652: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 735 op/s 2024-05-21T12:36:09.981 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:36:09.984 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:36:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:36:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: cluster: 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: health: HEALTH_WARN 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: 2 failed cephadm daemon(s) 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: services: 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: mon: 3 daemons, quorum a,c,b (age 17m) 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: mgr: x(active, since 18m) 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: osd: 8 osds: 8 up (since 14m), 8 in (since 14m) 2024-05-21T12:36:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: data: 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: pools: 2 pools, 33 pgs 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: objects: 485 objects, 1.0 GiB 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: usage: 3.3 GiB used, 712 GiB / 715 GiB avail 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: pgs: 33 active+clean 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: io: 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: client: 6.9 MiB/s rd, 6.7 MiB/s wr, 337 op/s rd, 324 op/s wr 2024-05-21T12:36:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:36:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.511 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.528 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.547 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.565 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.584 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.602 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.621 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.639 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.658 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:10 smithi072 ceph-mon[33912]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:36:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:10 smithi072 ceph-mon[33912]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:36:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:10 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/3344877481' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:36:10.676 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.694 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.713 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.730 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.749 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:10 smithi092 ceph-mon[33665]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:36:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:10 smithi092 ceph-mon[33665]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:36:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:10 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/3344877481' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:36:10.767 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.786 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.804 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.815 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:10 smithi045 ceph-mon[30165]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:36:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:10 smithi045 ceph-mon[30165]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:36:10.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:10 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/3344877481' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:36:10.822 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.841 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.896 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.914 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.933 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.951 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.969 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:10.987 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.005 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.023 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.041 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.060 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.078 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.096 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.115 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.133 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.151 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.170 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.188 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.206 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.224 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.242 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.260 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.279 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.297 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.315 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.333 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.352 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.370 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.388 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.406 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.425 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.443 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:36:11.444 INFO:tasks.nvmeof.[nvmeof.thrasher]:reviving nvmeof.a 2024-05-21T12:36:11.444 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Restarting nvmeof.nvmeof.a (starting--it wasn't running)... 2024-05-21T12:36:11.444 DEBUG:teuthology.orchestra.run.smithi045:nvmeof.nvmeof.a> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:36:11.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl start ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:36:11.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:11 smithi072 ceph-mon[33912]: pgmap v653: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 6.7 MiB/s wr, 661 op/s 2024-05-21T12:36:11.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:11 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/2466950063' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:36:11.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:11 smithi092 ceph-mon[33665]: pgmap v653: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 6.7 MiB/s wr, 661 op/s 2024-05-21T12:36:11.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:11 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/2466950063' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:36:11.753 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:11 smithi045 ceph-mon[30165]: pgmap v653: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 6.7 MiB/s wr, 661 op/s 2024-05-21T12:36:11.753 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:11 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/2466950063' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:36:11.753 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:11 smithi045 systemd[1]: Starting Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:36:12.011 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:11 smithi045 podman[75111]: 2024-05-21 12:36:11.656484334 +0000 UTC m=+0.019467327 image pull 30dd525ace30bc703cdf4a290b3be2b24f6ca7b74757424a10799eaf370636b4 quay.io/ceph/nvmeof:1.2 2024-05-21T12:36:12.011 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:11 smithi045 podman[75111]: 2024-05-21 12:36:11.784018908 +0000 UTC m=+0.147001894 container create f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.component=ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, release=, distribution-scope=public, io.spdk.version=24.01, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.nvmeof=None, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.build.makeflags=, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, vcs-type=git, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , name=ceph-nvmeof, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, vendor=Red Hat, Inc., io.spdk.build.pkgdep-args=--rbd, io.ceph.version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.git.branch=undefined, io.spdk.build.ceph-release=19.0.0, io.buildah.version=1.29.0, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, architecture=x86_64, io.spdk.build.target-arch=, io.openshift.wants=ceph,rbd, com.redhat.component=ubi9-container, org.centos.version=9.0-21.el9, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.summary=Ceph NVMe over Fabrics Gateway, build-date=2024-05-19 17:57:19 UTC, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.maintainer=Ceph Developers , summary=Ceph NVMe over Fabrics Gateway, version=1.2.9, url=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:36:12.252 INFO:tasks.nvmeof.[nvmeof.thrasher]:reviving nvmeof.b 2024-05-21T12:36:12.252 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.b:Restarting nvmeof.nvmeof.b (starting--it wasn't running)... 2024-05-21T12:36:12.252 DEBUG:teuthology.orchestra.run.smithi072:nvmeof.nvmeof.b> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:36:12.254 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl start ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:36:12.268 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 podman[75111]: 2024-05-21 12:36:12.144939973 +0000 UTC m=+0.507922967 container init f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.openshift.wants=ceph,rbd, org.centos.version=9.0-21.el9, io.spdk.build.pkgdep-args=--rbd, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.git.branch=undefined, url=https://github.com/ceph/ceph-nvmeof, release=, name=ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.component=ceph-nvmeof, io.spdk.build.target-arch=, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.ceph-release=19.0.0, distribution-scope=public, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, maintainer=Ceph Developers , io.ceph.maintainer=Ceph Developers , io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., version=1.2.9, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vendor=Red Hat, Inc., architecture=x86_64, com.redhat.component=ubi9-container, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.version=1.2.9, io.buildah.version=1.29.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Ceph NVMe over Fabrics Gateway, io.spdk.version=24.01, io.ceph.nvmeof=None, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.makeflags=, io.ceph.git.branch=tags/1.2.9, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, vcs-type=git, build-date=2024-05-19 17:57:19 UTC) 2024-05-21T12:36:12.268 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 podman[75111]: 2024-05-21 12:36:12.148104111 +0000 UTC m=+0.511087103 container start f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.openshift.wants=ceph,rbd, io.ceph.git.branch=tags/1.2.9, io.spdk.build.ceph-release=19.0.0, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, build-date=2024-05-19 17:57:19 UTC, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, architecture=x86_64, io.ceph.summary=Ceph NVMe over Fabrics Gateway, release=, io.spdk.build.pkgdep-args=--rbd, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, distribution-scope=public, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, org.centos.version=9.0-21.el9, io.spdk.build.makeflags=, io.ceph.component=ceph-nvmeof, name=ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, maintainer=Ceph Developers , io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, version=1.2.9, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.version=1.2.9, io.ceph.maintainer=Ceph Developers , io.spdk.version=24.01, io.spdk.git.repo=https://github.com/ceph/spdk.git, vcs-type=git, vendor=Red Hat, Inc., io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., com.redhat.component=ubi9-container, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.git.branch=undefined, io.spdk.build.target-arch=, io.ceph.url=https://github.com/ceph/ceph-nvmeof, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.buildah.version=1.29.0, summary=Ceph NVMe over Fabrics Gateway, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://github.com/ceph/ceph-nvmeof, io.ceph.nvmeof=None) 2024-05-21T12:36:12.268 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 bash[75111]: f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 2024-05-21T12:36:12.268 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 systemd[1]: Started Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:36:12.519 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO utils.py:258: Initialize gateway log level to "INFO" 2024-05-21T12:36:12.519 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO utils.py:271: Log files will be saved in /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a, using rotation 2024-05-21T12:36:12.519 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:78: Using NVMeoF gateway version 1.2.9 2024-05-21T12:36:12.519 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:81: Using SPDK version 24.01 2024-05-21T12:36:12.519 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:84: Using vstart cluster version based on 19.0.0 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:87: NVMeoF gateway built on: 2024-05-19 17:57:19 UTC 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:90: NVMeoF gateway Git repository: https://github.com/ceph/ceph-nvmeof 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:93: NVMeoF gateway Git branch: tags/1.2.9 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:96: NVMeoF gateway Git commit: 06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:102: SPDK Git repository: https://github.com/ceph/spdk.git 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:105: SPDK Git branch: undefined 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:108: SPDK Git commit: dade5b4c96a647e63ef96d9a5e05f3b64ea0598a 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:59: Using configuration file /src/ceph-nvmeof.conf 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:61: ====================================== Configuration file content ====================================== 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: # This file is generated by cephadm. 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: [gateway] 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: name = client.nvmeof.nvmeof.a 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: group = 2024-05-21T12:36:12.520 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: addr = 172.21.15.45 2024-05-21T12:36:12.521 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: port = 5500 2024-05-21T12:36:12.521 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: enable_auth = False 2024-05-21T12:36:12.521 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: state_update_notify = True 2024-05-21T12:36:12.521 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: state_update_interval_sec = 5 2024-05-21T12:36:12.524 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: enable_spdk_discovery_controller = False 2024-05-21T12:36:12.524 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: enable_prometheus_exporter = True 2024-05-21T12:36:12.524 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: prometheus_exporter_ssl = False 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: prometheus_port = 10008 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: verify_nqns = True 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: omap_file_lock_duration = 20 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: omap_file_lock_retries = 30 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: omap_file_lock_retry_sleep_interval = 1.0 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: omap_file_update_reloads = 10 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: allowed_consecutive_spdk_ping_failures = 1 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: spdk_ping_interval_in_seconds = 2.0 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: ping_spdk_under_lock = False 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: enable_monitor_client = True 2024-05-21T12:36:12.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: [gateway-logs] 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: log_level = INFO 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: log_files_enabled = True 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: log_files_rotation_enabled = True 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: verbose_log_messages = True 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: max_log_file_size_in_mb = 10 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: max_log_files_count = 20 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: max_log_directory_backups = 10 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: log_directory = /var/log/ceph/ 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: [discovery] 2024-05-21T12:36:12.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: addr = 172.21.15.45 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: port = 8009 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: [ceph] 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: pool = mypool 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: config_file = /etc/ceph/ceph.conf 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: id = nvmeof.nvmeof.a 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: [mtls] 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: server_key = ./server.key 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: client_key = ./client.key 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: server_cert = ./server.crt 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: client_cert = ./client.crt 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: [spdk] 2024-05-21T12:36:12.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: tgt_path = /usr/local/bin/nvmf_tgt 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: rpc_socket_dir = /var/tmp/ 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: rpc_socket_name = spdk.sock 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: timeout = 60.0 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: bdevs_per_cluster = 32 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: log_level = WARNING 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: conn_retries = 10 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: transports = tcp 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: transport_tcp_options = {"in_capsule_data_size": 8192, "max_io_qpairs_per_ctrlr": 7} 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: [monitor] 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:65: timeout = 1.0 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO config.py:66: ======================================================================================================== 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:90: Starting gateway client.nvmeof.nvmeof.a 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:157: Starting serve, monitor client version: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:36:12.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:236: Starting /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.a --gateway-address 172.21.15.45:5500 --gateway-pool mypool --gateway-group --monitor-group-address 172.21.15.45:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.a -k /etc/ceph/keyring 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:240: monitor client process id: 19 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:146: MonitorGroup server is listening on 172.21.15.45:5499 for group id 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.501+0000 7fd5e660b5c0 0 nvmeofgw int NVMeofGwMonitorClient::init() 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.501+0000 7fd5e660b5c0 0 nvmeofgw int NVMeofGwMonitorClient::init() gateway name: client.nvmeof.nvmeof.a pool:mypool group: address: 172.21.15.45:5500 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.501+0000 7fd5e660b5c0 0 nvmeofgw int NVMeofGwMonitorClient::init() nvmeof Registered monc callback 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.504+0000 7fd5e660b5c0 0 nvmeofgw int NVMeofGwMonitorClient::init() monc.authentication done 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.504+0000 7fd5e660b5c0 0 nvmeofgw void NVMeofGwMonitorClient::tick() 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.505+0000 7fd5e660b5c0 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.505+0000 7fd5e660b5c0 0 nvmeofgw void NVMeofGwMonitorClient::send_beacon() sending beacon as gid 15294 availability 0 osdmap_epoch 0 gwmap_epoch 0 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.505+0000 7fd5e660b5c0 0 nvmeofgw int NVMeofGwMonitorClient::init() Complete. 2024-05-21T12:36:12.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.505+0000 7fd5e25a8640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.505+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 127 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.505+0000 7fd5e25a8640 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.506+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_group_id: 0 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:135: Gateway client.nvmeof.nvmeof.a group id=0 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:149: Stopping the MonitorGroup server... 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.531 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.532 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.533 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.508+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.510+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:152: The MonitorGroup gRPC server stopped... 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.512+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:349: SPDK Target Path: /usr/local/bin/nvmf_tgt 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:351: SPDK Socket: /var/tmp/spdk.sock 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:357: Starting /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.513+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:370: SPDK process id: 69 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:371: Attempting to initialize SPDK: rpc_socket: /var/tmp/spdk.sock, conn_retries: 300, timeout: 60.0 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: INFO: Setting log level to WARNING 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO client.py:110: Setting log level to WARNING 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.515+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.517+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.518+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.520+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.522+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.536 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.524+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.556 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:12 smithi072 systemd[1]: Starting Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:36:12.769 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.525+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.769 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.527+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.769 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.528+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:12.529786] Starting SPDK v24.01 / DPDK 23.11.0 initialization... 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:12.529867] [ DPDK EAL parameters: nvmf --no-shconf -c 0x1 --no-pci --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --iova-mode=pa --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid69 ] 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.530+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.531+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.533+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.535+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.536+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.538+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.539+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.541+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.542+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.770 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.544+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.771 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.545+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.771 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.547+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.771 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.548+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.771 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.550+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.551+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.553+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.554+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.556+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.558+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.559+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.560+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.562+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.563+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.565+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.566+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.772 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.568+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.569+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.571+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.572+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.574+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.575+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.577+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.579+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.580+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.582+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.584+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.585+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.773 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.587+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.589+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.590+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.592+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.593+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.595+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.597+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.598+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.600+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.601+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.603+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.774 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.604+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.606+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.608+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.609+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.611+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.612+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.614+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.615+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.617+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.618+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.620+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.775 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.621+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.776 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.623+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.624+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.626+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.627+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.629+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.630+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.632+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.633+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.635+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.637+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.638+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.640+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.641+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.777 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.643+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.778 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.644+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.778 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:12.645729] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 2024-05-21T12:36:12.778 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.646+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.778 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.647+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.778 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.649+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.778 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.651+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.778 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.653+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.778 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.654+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.656+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.658+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.660+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.661+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.663+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.664+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.666+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.668+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.670+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.779 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.672+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.674+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.676+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.677+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.679+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.681+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.682+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.684+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.685+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.687+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.688+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.780 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.690+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.692+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.693+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.695+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.697+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.698+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.700+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.702+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.704+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.706+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:12.706590] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.708+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.710+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.781 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.711+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.713+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.715+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.717+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.719+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.721+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.722+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.724+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.725+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.727+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.729+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.731+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.782 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.733+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.735+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.737+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.740+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.742+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.744+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.746+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.747+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.749+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.751+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.752+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.754+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.755+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.783 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.757+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.758+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.760+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.761+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.763+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.764+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.766+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.767+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.769+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.770+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.772+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.774+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.784 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.775+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:12.841 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:12 smithi072 podman[56956]: 2024-05-21 12:36:12.460388015 +0000 UTC m=+0.020424850 image pull 30dd525ace30bc703cdf4a290b3be2b24f6ca7b74757424a10799eaf370636b4 quay.io/ceph/nvmeof:1.2 2024-05-21T12:36:12.841 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:12 smithi072 podman[56956]: 2024-05-21 12:36:12.594305262 +0000 UTC m=+0.154342082 container create 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.pkgdep-args=--rbd, name=ceph-nvmeof, io.spdk.git.repo=https://github.com/ceph/spdk.git, architecture=x86_64, io.ceph.url=https://github.com/ceph/ceph-nvmeof, com.redhat.component=ubi9-container, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, vendor=Red Hat, Inc., summary=Ceph NVMe over Fabrics Gateway, url=https://github.com/ceph/ceph-nvmeof, io.ceph.component=ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, maintainer=Ceph Developers , io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.version=1.2.9, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, release=, io.ceph.maintainer=Ceph Developers , io.spdk.build.makeflags=, io.openshift.wants=ceph,rbd, io.spdk.build.ceph-release=19.0.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.nvmeof=None, build-date=2024-05-19 17:57:19 UTC, distribution-scope=public, vcs-type=git, io.spdk.build.target-arch=, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.version=24.01, io.spdk.git.branch=undefined, io.buildah.version=1.29.0, version=1.2.9, org.centos.version=9.0-21.el9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a) 2024-05-21T12:36:12.959 INFO:tasks.nvmeof.[nvmeof.thrasher]:waiting for 67 secs before thrashing 2024-05-21T12:36:13.023 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.777+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.023 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.778+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.023 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.780+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.781+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.783+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.785+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.786+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.788+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.790+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.791+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.793+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.794+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.796+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.798+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.024 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.799+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.025 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.801+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.025 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.802+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.804+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.806+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.807+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.809+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.810+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.812+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.814+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.815+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.817+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.026 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.819+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.820+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.822+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.823+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.825+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.827+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.828+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.830+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.831+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.833+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.835+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.836+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.838+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.027 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.839+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.841+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.842+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.844+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.845+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.847+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.848+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.850+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.852+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.853+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.855+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.856+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.858+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.028 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.859+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.861+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.863+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.864+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.866+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.867+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.869+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.871+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.872+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.874+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.875+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.877+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.879+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.029 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.880+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.882+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.883+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.885+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.887+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.888+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.890+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.892+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.893+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.895+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.896+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.898+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.900+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.030 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.901+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.903+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.904+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.906+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.908+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.909+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.911+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.912+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.914+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.916+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.917+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:12.918753] tcp.c: 659:nvmf_tcp_create: *NOTICE*: *** TCP Transport Init *** 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.919+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.031 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.920+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.922+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.924+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.925+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.927+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.928+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.930+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.032 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:268: Discovery service process id: 843 2024-05-21T12:36:13.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:264: Starting ceph nvmeof discovery service 2024-05-21T12:36:13.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.932+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.934+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.033 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.935+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.937+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.939+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.941+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.943+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.945+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO cephutils.py:74: Connected to Ceph with version "19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)" 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:86: Requested huge pages count is 2048 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.947+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:105: Actual huge pages count is 4096 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:190: NVMeoF bdevs per cluster: 32 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO discovery.py:322: log pages info from omap: nvmeof.state 2024-05-21T12:36:13.034 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.949+0000 7fd5e25a8640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO server.py:195: Prometheus endpoint is enabled 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO prometheus.py:98: Prometheus exporter running in http mode, listening on port 10008 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO discovery.py:330: discovery addr: 172.21.15.45 port: 8009 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:2648: Received request to get gateway's info 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO prometheus.py:131: Stats for all bdevs will be provided 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 1 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.035 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 2 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 3 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 4 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 5 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 6 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 7 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 8 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 9 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 10 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 11 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 12 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 13 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 14 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 15 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 16 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 1 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 2 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 3 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 4 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 5 2024-05-21T12:36:13.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 6 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 7 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 8 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.042 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 9 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 10 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 11 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.043 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 12 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 13 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 14 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.044 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 15 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 16 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 1 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.045 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 2 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 3 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 4 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.046 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 5 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 6 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 7 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 8 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.047 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 9 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 10 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 11 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.048 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 12 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 13 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 14 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 15 2024-05-21T12:36:13.049 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 16 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: INACCESSIBLE 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: ], , client address: IPv4 172.21.15.45:43556 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode2 not found 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.972+0000 7fd5e25a8640 0 ms_deliver_dispatch: unhandled message 0x559ce1899180 nvmeofgwmap magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.972+0000 7fd5e25a8640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 4 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.972+0000 7fd5e25a8640 0 ms_deliver_dispatch: unhandled message 0x559ce1889d40 mon_map magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: 2024-05-21T12:36:12.972+0000 7fd5e25a8640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 41 2024-05-21T12:36:13.050 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:13.051 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:12] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.051 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.008283] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.45:0/864788791, rc 0 2024-05-21T12:36:13.051 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.008394] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.45:0/864788791 2024-05-21T12:36:13.051 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.45:0/864788791' anagrp=2 2024-05-21T12:36:13.051 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:36:13.051 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.023516] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:36:13.051 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c 2024-05-21T12:36:13.113 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:12 smithi072 podman[56956]: 2024-05-21 12:36:12.842095603 +0000 UTC m=+0.402132423 container init 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.buildah.version=1.29.0, io.spdk.build.target-arch=, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, release=, org.centos.version=9.0-21.el9, io.spdk.version=24.01, io.openshift.wants=ceph,rbd, com.redhat.component=ubi9-container, io.spdk.build.pkgdep-args=--rbd, vcs-type=git, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, build-date=2024-05-19 17:57:19 UTC, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, maintainer=Ceph Developers , io.spdk.git.repo=https://github.com/ceph/spdk.git, version=1.2.9, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.build.makeflags=, io.ceph.summary=Ceph NVMe over Fabrics Gateway, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, summary=Ceph NVMe over Fabrics Gateway, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, name=ceph-nvmeof, io.ceph.nvmeof=None, io.ceph.maintainer=Ceph Developers , io.spdk.build.ceph-release=19.0.0, url=https://github.com/ceph/ceph-nvmeof, io.ceph.component=ceph-nvmeof, io.ceph.version=1.2.9, io.ceph.url=https://github.com/ceph/ceph-nvmeof, vendor=Red Hat, Inc., io.spdk.git.branch=undefined, distribution-scope=public, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.branch=tags/1.2.9, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:36:13.114 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:12 smithi072 podman[56956]: 2024-05-21 12:36:12.845571302 +0000 UTC m=+0.405608131 container start 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.ceph.url=https://github.com/ceph/ceph-nvmeof, distribution-scope=public, io.ceph.git.branch=tags/1.2.9, architecture=x86_64, io.ceph.nvmeof=None, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.spdk.build.pkgdep-args=--rbd, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.version=24.01, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.openshift.wants=ceph,rbd, com.redhat.component=ubi9-container, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., name=ceph-nvmeof, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vendor=Red Hat, Inc., io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.git.branch=undefined, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.version=1.2.9, release=, io.spdk.build.ceph-release=19.0.0, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, summary=Ceph NVMe over Fabrics Gateway, io.buildah.version=1.29.0, io.spdk.build.target-arch=, io.ceph.component=ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, org.centos.version=9.0-21.el9, build-date=2024-05-19 17:57:19 UTC, io.ceph.maintainer=Ceph Developers , io.spdk.build.makeflags=, vcs-type=git, url=https://github.com/ceph/ceph-nvmeof, version=1.2.9) 2024-05-21T12:36:13.114 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:12 smithi072 bash[56956]: 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:36:13.114 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:12 smithi072 systemd[1]: Started Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:36:13.114 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO utils.py:258: Initialize gateway log level to "INFO" 2024-05-21T12:36:13.114 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO utils.py:271: Log files will be saved in /var/log/ceph/nvmeof-client.nvmeof.nvmeof.b, using rotation 2024-05-21T12:36:13.114 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:78: Using NVMeoF gateway version 1.2.9 2024-05-21T12:36:13.114 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:81: Using SPDK version 24.01 2024-05-21T12:36:13.114 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:84: Using vstart cluster version based on 19.0.0 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:87: NVMeoF gateway built on: 2024-05-19 17:57:19 UTC 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:90: NVMeoF gateway Git repository: https://github.com/ceph/ceph-nvmeof 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:93: NVMeoF gateway Git branch: tags/1.2.9 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:96: NVMeoF gateway Git commit: 06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:102: SPDK Git repository: https://github.com/ceph/spdk.git 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:105: SPDK Git branch: undefined 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:108: SPDK Git commit: dade5b4c96a647e63ef96d9a5e05f3b64ea0598a 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:59: Using configuration file /src/ceph-nvmeof.conf 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:61: ====================================== Configuration file content ====================================== 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: # This file is generated by cephadm. 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: [gateway] 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: name = client.nvmeof.nvmeof.b 2024-05-21T12:36:13.115 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: group = 2024-05-21T12:36:13.116 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: addr = 172.21.15.72 2024-05-21T12:36:13.116 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: port = 5500 2024-05-21T12:36:13.274 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:13.274 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.274 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.046368] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.45:0/379124416, rc 0 2024-05-21T12:36:13.274 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.046480] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.45:0/379124416 2024-05-21T12:36:13.274 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.45:0/379124416' anagrp=1 2024-05-21T12:36:13.274 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:36:13.274 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.061239] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:36:13.274 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad 2024-05-21T12:36:13.275 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:13.275 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.275 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:36:13.275 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.090671] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:36:13.275 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 2024-05-21T12:36:13.275 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.099123] nvmf_rpc.c: 275:rpc_nvmf_get_subsystems: *WARNING*: rpc_nvmf_get_subsystems: deprecated feature listener.transport is deprecated in favor of trtype to be removed in v24.05 2024-05-21T12:36:13.275 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.133119] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.45:0/3594543521, rc 0 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.133831] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.45:0/3594543521 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.45:0/3594543521' anagrp=3 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.150185] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.244395] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:36:13.276 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 2024-05-21T12:36:13.277 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:13.277 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.277 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:36:13.277 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.265352] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:36:13.277 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f 2024-05-21T12:36:13.277 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:13.277 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.277 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:36:13.365 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-mon[33912]: pgmap v654: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-05-21T12:36:13.365 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/3120632535' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: enable_auth = False 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: state_update_notify = True 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: state_update_interval_sec = 5 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: enable_spdk_discovery_controller = False 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: enable_prometheus_exporter = True 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: prometheus_exporter_ssl = False 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: prometheus_port = 10008 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: verify_nqns = True 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: omap_file_lock_duration = 20 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: omap_file_lock_retries = 30 2024-05-21T12:36:13.366 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: omap_file_lock_retry_sleep_interval = 1.0 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: omap_file_update_reloads = 10 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: allowed_consecutive_spdk_ping_failures = 1 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: spdk_ping_interval_in_seconds = 2.0 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: ping_spdk_under_lock = False 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: enable_monitor_client = True 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: [gateway-logs] 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: log_level = INFO 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: log_files_enabled = True 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: log_files_rotation_enabled = True 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: verbose_log_messages = True 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: max_log_file_size_in_mb = 10 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: max_log_files_count = 20 2024-05-21T12:36:13.367 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: max_log_directory_backups = 10 2024-05-21T12:36:13.370 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: log_directory = /var/log/ceph/ 2024-05-21T12:36:13.370 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: 2024-05-21T12:36:13.370 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: [discovery] 2024-05-21T12:36:13.370 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: addr = 172.21.15.72 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: port = 8009 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: [ceph] 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: pool = mypool 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: config_file = /etc/ceph/ceph.conf 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: id = nvmeof.nvmeof.b 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: [mtls] 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: server_key = ./server.key 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: client_key = ./client.key 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: server_cert = ./server.crt 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: client_cert = ./client.crt 2024-05-21T12:36:13.371 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: [spdk] 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: tgt_path = /usr/local/bin/nvmf_tgt 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: rpc_socket_dir = /var/tmp/ 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: rpc_socket_name = spdk.sock 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: timeout = 60.0 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: bdevs_per_cluster = 32 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: log_level = WARNING 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: conn_retries = 10 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: transports = tcp 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: transport_tcp_options = {"in_capsule_data_size": 8192, "max_io_qpairs_per_ctrlr": 7} 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: [monitor] 2024-05-21T12:36:13.372 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:65: timeout = 1.0 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO config.py:66: ======================================================================================================== 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:90: Starting gateway client.nvmeof.nvmeof.b 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:157: Starting serve, monitor client version: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:236: Starting /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.b --gateway-address 172.21.15.72:5500 --gateway-pool mypool --gateway-group --monitor-group-address 172.21.15.72:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.b -k /etc/ceph/keyring 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:240: monitor client process id: 19 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:146: MonitorGroup server is listening on 172.21.15.72:5499 for group id 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.196+0000 7f5de13465c0 0 nvmeofgw int NVMeofGwMonitorClient::init() 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.196+0000 7f5de13465c0 0 nvmeofgw int NVMeofGwMonitorClient::init() gateway name: client.nvmeof.nvmeof.b pool:mypool group: address: 172.21.15.72:5500 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.196+0000 7f5de13465c0 0 nvmeofgw int NVMeofGwMonitorClient::init() nvmeof Registered monc callback 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.199+0000 7f5de13465c0 0 nvmeofgw int NVMeofGwMonitorClient::init() monc.authentication done 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.199+0000 7f5de13465c0 0 nvmeofgw void NVMeofGwMonitorClient::tick() 2024-05-21T12:36:13.373 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.200+0000 7f5de13465c0 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.200+0000 7f5de13465c0 0 nvmeofgw void NVMeofGwMonitorClient::send_beacon() sending beacon as gid 25088 availability 0 osdmap_epoch 0 gwmap_epoch 0 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.200+0000 7f5de13465c0 0 nvmeofgw int NVMeofGwMonitorClient::init() Complete. 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.200+0000 7f5ddd2e3640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.200+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 127 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.200+0000 7f5ddd2e3640 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.200+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_group_id: 1 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:135: Gateway client.nvmeof.nvmeof.b group id=1 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:149: Stopping the MonitorGroup server... 2024-05-21T12:36:13.374 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.375 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.376 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.377 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.378 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.202+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.379 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.203+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.379 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.204+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.379 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:152: The MonitorGroup gRPC server stopped... 2024-05-21T12:36:13.379 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.206+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.379 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:349: SPDK Target Path: /usr/local/bin/nvmf_tgt 2024-05-21T12:36:13.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:351: SPDK Socket: /var/tmp/spdk.sock 2024-05-21T12:36:13.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:357: Starting /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:36:13.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.207+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.209+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:370: SPDK process id: 70 2024-05-21T12:36:13.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:371: Attempting to initialize SPDK: rpc_socket: /var/tmp/spdk.sock, conn_retries: 300, timeout: 60.0 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: INFO: Setting log level to WARNING 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO client.py:110: Setting log level to WARNING 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.210+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.212+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.214+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.215+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.217+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.219+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.220+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.222+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.225114] Starting SPDK v24.01 / DPDK 23.11.0 initialization... 2024-05-21T12:36:13.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.225202] [ DPDK EAL parameters: nvmf --no-shconf -c 0x1 --no-pci --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --iova-mode=pa --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid70 ] 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.224+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.225+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.227+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.228+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.230+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.231+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.233+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.234+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.236+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.238+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.239+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.241+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.243+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.244+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.246+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.247+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.249+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.250+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.252+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.253+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.255+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.256+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.258+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.259+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.261+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.262+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.264+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.265+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.267+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.268+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.270+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.271+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.273+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.274+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.276+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.277+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.279+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.280+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.282+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.283+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.285+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.286+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.288+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.289+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.291+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.292+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.294+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.295+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.297+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.298+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.300+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.301+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.303+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.305+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.306+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.308+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.309+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.311+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.312+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.314+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.315+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.317+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.318+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.386 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.320+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.321+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.323+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.324+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.326+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.327+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.329+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.331+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.332+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.334+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.335+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.338+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.339736] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.340+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.342+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.387 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.344+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.345+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.347+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.349+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.350+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.352+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.353+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.355+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.357+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.359+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.360+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.362+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.364+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.367+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.368+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.388 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.370+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.389 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.372+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-mon[30165]: pgmap v654: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-05-21T12:36:13.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/3120632535' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:36:13.524 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.286265] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.309050] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.331766] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea 2024-05-21T12:36:13.525 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:13.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:36:13.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.353984] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:36:13.526 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.374462] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.397050] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.527 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.418290] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.440440] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.460823] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:36:13.528 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.483288] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.504600] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:36:13.529 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.524886] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:36:13.530 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.374+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.375+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.377+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.379+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.380+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.382+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.384+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.385+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.387+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.616 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.389+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.617 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.390+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.617 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.392935] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 2024-05-21T12:36:13.617 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.392+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.617 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.394+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.617 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.396+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.617 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.398+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.400+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.402+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.404+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.405+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.407+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.408+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.410+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.412+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.414+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.618 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.416+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.418+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.420+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.422+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.424+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.426+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.428+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.430+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.432+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.434+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.436+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.437+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.439+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.619 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.440+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.442+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.443+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.445+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.446+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.448+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.449+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.451+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.453+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.454+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.456+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.457+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.459+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.620 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.460+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.462+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.464+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.465+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.467+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.468+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.470+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.471+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.473+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.474+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.476+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.477+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.479+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.481+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.621 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.483+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.622 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.484+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.486+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.487+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.489+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.490+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.492+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.493+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.495+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.496+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.498+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.499+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.501+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.623 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.502+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.504+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.505+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.507+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.508+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.510+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.511+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.513+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.514+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.516+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.518+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.519+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.521+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.522+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.624 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.524+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.525+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.527+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.528+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.530+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.531+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.533+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.534+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.536+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.537+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.539+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.540+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.625 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.542+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.543+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.545+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.546+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.548+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.549+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.551+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.552+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.554+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.555+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.557+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.558+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.626 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.560+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.562+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.563+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.565+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.566+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.568+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.570+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.571+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.573+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.574+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.576+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.577+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.578+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.580+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.627 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.581+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.583+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.584+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.586+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.587+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.589+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.590+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.592+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.593+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.595+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.597+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.598+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.599+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.628 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.601+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.603+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.604+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.605+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.607+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.608+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.610+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.612+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.613735] tcp.c: 659:nvmf_tcp_create: *NOTICE*: *** TCP Transport Init *** 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.613+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.615+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.617+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.618+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.629 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.620+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:13 smithi092 ceph-mon[33665]: pgmap v654: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-05-21T12:36:13.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:13 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/3120632535' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:36:13.785 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.545472] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.567792] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:13.786 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.787 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:36:13.787 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.589300] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:36:13.787 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 2024-05-21T12:36:13.787 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:13.787 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.787 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.610347] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.632192] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:36:13.788 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.654027] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.676039] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.696578] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:13.789 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.718940] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.742230] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.790 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:36:13.791 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.765050] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:36:13.791 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 2024-05-21T12:36:13.791 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:13.791 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.791 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:36:13.791 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.786055] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:36:13.791 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:268: Discovery service process id: 852 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:264: Starting ceph nvmeof discovery service 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.625+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.627+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.629+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.631+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.633+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.635+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.867 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.637+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.868 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.639+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.868 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO cephutils.py:74: Connected to Ceph with version "19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)" 2024-05-21T12:36:13.868 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:86: Requested huge pages count is 2048 2024-05-21T12:36:13.868 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.640+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.868 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:105: Actual huge pages count is 4096 2024-05-21T12:36:13.868 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:190: NVMeoF bdevs per cluster: 32 2024-05-21T12:36:13.868 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:36:13.868 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO discovery.py:322: log pages info from omap: nvmeof.state 2024-05-21T12:36:13.869 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.642+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:36:13.869 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:36:13.869 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO server.py:195: Prometheus endpoint is enabled 2024-05-21T12:36:13.869 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO discovery.py:330: discovery addr: 172.21.15.72 port: 8009 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO prometheus.py:98: Prometheus exporter running in http mode, listening on port 10008 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 2 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.870 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 4 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 5 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 6 2024-05-21T12:36:13.871 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 7 2024-05-21T12:36:13.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 8 2024-05-21T12:36:13.872 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 9 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 10 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.873 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 11 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 12 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 13 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.874 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 14 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 15 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 16 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:36:13.875 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 2 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.876 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 4 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 5 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 6 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 7 2024-05-21T12:36:13.877 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 8 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 9 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 10 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.878 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 11 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 12 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 13 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.879 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 14 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 15 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 16 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 2 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 4 2024-05-21T12:36:13.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 5 2024-05-21T12:36:13.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 6 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 7 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 8 2024-05-21T12:36:13.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 9 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 10 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 11 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 12 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 13 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 14 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.886 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 15 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 16 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:47656 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode1 not found 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode2 not found 2024-05-21T12:36:13.887 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:2648: Received request to get gateway's info 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.657+0000 7f5ddd2e3640 0 ms_deliver_dispatch: unhandled message 0x5583f1043180 nvmeofgwmap magic: 0 from mon.2 v2:172.21.15.72:3300/0 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.657+0000 7f5ddd2e3640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 4 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.657+0000 7f5ddd2e3640 0 ms_deliver_dispatch: unhandled message 0x5583f1033d40 mon_map magic: 0 from mon.2 v2:172.21.15.72:3300/0 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO prometheus.py:131: Stats for all bdevs will be provided 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.658+0000 7f5ddd2e3640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 41 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.658+0000 7f5ddd2e3640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.658+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 128 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 128 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 128 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.658+0000 7f5ddd2e3640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) got_old_gw_state: NvmeGwState { group id: 1 gw_map_epoch 127 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }got_new_gw_state: NvmeGwState { group id: 1 gw_map_epoch 128 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,58,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: 2024-05-21T12:36:13.658+0000 7f5ddd2e3640 0 ms_deliver_dispatch: unhandled message 0x5583f1043500 nvmeofgwmap magic: 0 from mon.2 v2:172.21.15.72:3300/0 2024-05-21T12:36:13.888 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.702810] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.72:0/2012793163, rc 0 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.703394] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.72:0/2012793163 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.72:0/2012793163' anagrp=2 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.718996] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.745228] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.72:0/3603239213, rc 0 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.745354] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.72:0/3603239213 2024-05-21T12:36:13.889 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.72:0/3603239213' anagrp=1 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.760258] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.772936] nvmf_rpc.c: 275:rpc_nvmf_get_subsystems: *WARNING*: rpc_nvmf_get_subsystems: deprecated feature listener.transport is deprecated in favor of trtype to be removed in v24.05 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.808194] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.72:0/2138234012, rc 0 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.808523] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.72:0/2138234012 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.72:0/2138234012' anagrp=3 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.822214] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:36:13.890 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 2024-05-21T12:36:13.891 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:13.891 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:13.891 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:36:13.891 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.867265] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:36:13.891 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 2024-05-21T12:36:14.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:36:14.036 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.807095] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.829585] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.851690] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:36:14.037 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b 2024-05-21T12:36:14.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:36:14.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.872935] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:36:14.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 2024-05-21T12:36:14.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.038 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.896654] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.920059] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.941433] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.039 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.963699] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:13.985068] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:13 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.006218] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a 2024-05-21T12:36:14.040 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:36:14.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.027513] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:36:14.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 2024-05-21T12:36:14.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:14.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.041 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:36:14.125 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.936952] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.958768] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.126 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.127 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:36:14.127 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:13.980584] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:36:14.127 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 2024-05-21T12:36:14.127 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:13 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:13] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.002321] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.022846] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:36:14.128 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.044110] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.064284] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:36:14.129 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.085742] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.106466] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.126084] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:36:14.130 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a 2024-05-21T12:36:14.291 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.047888] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:36:14.291 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.067914] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.090336] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:14.292 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:36:14.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.111518] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:36:14.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 2024-05-21T12:36:14.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.293 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.132482] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.154836] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.294 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.177706] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.198286] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.220554] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:36:14.295 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.241146] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.263194] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:36:14.296 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 2024-05-21T12:36:14.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:36:14.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.283003] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:36:14.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e 2024-05-21T12:36:14.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:14.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.297 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:36:14.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/809760610' entity='client.nvmeof.nvmeof.b' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:36:14.379 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.379 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.379 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.147386] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.167725] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.189843] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:36:14.380 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.210567] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.230741] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:36:14.381 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.250936] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.273389] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f 2024-05-21T12:36:14.382 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.294372] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.315095] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 2024-05-21T12:36:14.383 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.338438] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.359253] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 2024-05-21T12:36:14.384 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:36:14.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.379666] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:36:14.385 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 2024-05-21T12:36:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/809760610' entity='client.nvmeof.nvmeof.b' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:36:14.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.304465] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.325653] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.346559] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.367537] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.387772] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.408602] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:36:14.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.429710] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: None 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: None 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: None 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: None 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:14.466260] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.45 port 4420 *** 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: None 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: None 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: None 2024-05-21T12:36:14.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: None 2024-05-21T12:36:14.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: None 2024-05-21T12:36:14.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: None 2024-05-21T12:36:14.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: None 2024-05-21T12:36:14.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:14 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:14] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: None 2024-05-21T12:36:14.630 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:14.630 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.630 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:36:14.630 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.400998] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:36:14.630 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 2024-05-21T12:36:14.630 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.630 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.630 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:36:14.631 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.422172] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:36:14.631 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 2024-05-21T12:36:14.631 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:36:14.631 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.631 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:36:14.631 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.443461] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:36:14.631 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.464311] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:36:14.632 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.484793] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.505859] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.526424] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 2024-05-21T12:36:14.633 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.546844] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.567450] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.634 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.587851] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.608478] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:36:14.635 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.629559] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:36:14.636 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 2024-05-21T12:36:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:14 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/809760610' entity='client.nvmeof.nvmeof.b' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:36:14.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:14.880 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.650982] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.672181] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:14.881 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.693596] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.714642] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.882 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.734875] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.756990] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:36:14.883 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.776781] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.796970] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.884 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:36:14.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.818517] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:36:14.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a 2024-05-21T12:36:14.885 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:14.886 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 167.2k 160.8k 0.0k 1.6M 1.6M 0.0k nvme2n1 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 204.8k 153.2k 0.0k 2.0M 1.5M 0.0k nvme2n10 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 134.0k 120.4k 0.0k 1.3M 1.2M 0.0k nvme2n11 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 111.6k 158.8k 0.0k 1.1M 1.6M 0.0k nvme2n12 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 160.0k 187.2k 0.0k 1.6M 1.8M 0.0k nvme2n13 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 138.8k 139.2k 0.0k 1.4M 1.4M 0.0k nvme2n14 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 132.8k 178.0k 0.0k 1.3M 1.7M 0.0k nvme2n15 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 131.6k 144.4k 0.0k 1.3M 1.4M 0.0k nvme2n16 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 127.2k 115.2k 0.0k 1.2M 1.1M 0.0k nvme2n17 2024-05-21T12:36:14.887 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 174.0k 157.6k 0.0k 1.7M 1.5M 0.0k nvme2n18 2024-05-21T12:36:14.888 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 126.0k 133.6k 0.0k 1.2M 1.3M 0.0k nvme2n19 2024-05-21T12:36:14.888 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 172.0k 134.4k 0.0k 1.7M 1.3M 0.0k nvme2n2 2024-05-21T12:36:14.888 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 189.2k 139.6k 0.0k 1.8M 1.4M 0.0k nvme2n20 2024-05-21T12:36:14.888 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 196.8k 130.8k 0.0k 1.9M 1.3M 0.0k nvme2n3 2024-05-21T12:36:14.888 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 196.4k 177.6k 0.0k 1.9M 1.7M 0.0k nvme2n4 2024-05-21T12:36:14.888 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 141.2k 196.4k 0.0k 1.4M 1.9M 0.0k nvme2n5 2024-05-21T12:36:14.888 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 176.8k 196.8k 0.0k 1.7M 1.9M 0.0k nvme2n6 2024-05-21T12:36:14.888 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 164.4k 158.0k 0.0k 1.6M 1.5M 0.0k nvme2n7 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 151.6k 172.4k 0.0k 1.5M 1.7M 0.0k nvme2n8 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 132.8k 168.4k 0.0k 1.3M 1.6M 0.0k nvme2n9 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 182.0k 157.2k 0.0k 1.8M 1.5M 0.0k nvme5n1 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 144.4k 110.0k 0.0k 1.4M 1.1M 0.0k nvme5n10 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 158.8k 175.2k 0.0k 1.6M 1.7M 0.0k nvme5n11 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 149.6k 145.2k 0.0k 1.5M 1.4M 0.0k nvme5n12 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 135.2k 145.6k 0.0k 1.3M 1.4M 0.0k nvme5n13 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 115.6k 129.6k 0.0k 1.1M 1.3M 0.0k nvme5n14 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 143.6k 109.2k 0.0k 1.4M 1.1M 0.0k nvme5n15 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 177.2k 176.0k 0.0k 1.7M 1.7M 0.0k nvme5n16 2024-05-21T12:36:14.889 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 144.4k 145.6k 0.0k 1.4M 1.4M 0.0k nvme5n17 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 192.8k 141.2k 0.0k 1.9M 1.4M 0.0k nvme5n18 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 110.8k 149.2k 0.0k 1.1M 1.5M 0.0k nvme5n19 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 183.6k 159.6k 0.0k 1.8M 1.6M 0.0k nvme5n2 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 143.2k 148.8k 0.0k 1.4M 1.5M 0.0k nvme5n20 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 101.6k 115.6k 0.0k 1016.0k 1.1M 0.0k nvme5n3 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 112.0k 145.2k 0.0k 1.1M 1.4M 0.0k nvme5n4 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 182.8k 182.4k 0.0k 1.8M 1.8M 0.0k nvme5n5 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 138.0k 187.6k 0.0k 1.3M 1.8M 0.0k nvme5n6 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 175.6k 180.4k 0.0k 1.7M 1.8M 0.0k nvme5n7 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 156.0k 120.8k 0.0k 1.5M 1.2M 0.0k nvme5n8 2024-05-21T12:36:14.890 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 138.4k 190.0k 0.0k 1.4M 1.9M 0.0k nvme5n9 2024-05-21T12:36:14.891 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 144.4k 168.0k 0.0k 1.4M 1.6M 0.0k nvme8n1 2024-05-21T12:36:14.891 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 158.4k 144.4k 0.0k 1.5M 1.4M 0.0k nvme8n10 2024-05-21T12:36:14.891 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 151.2k 149.2k 0.0k 1.5M 1.5M 0.0k nvme8n11 2024-05-21T12:36:14.891 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 148.8k 175.6k 0.0k 1.5M 1.7M 0.0k nvme8n12 2024-05-21T12:36:14.891 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 138.8k 150.4k 0.0k 1.4M 1.5M 0.0k nvme8n13 2024-05-21T12:36:14.891 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 174.0k 149.2k 0.0k 1.7M 1.5M 0.0k nvme8n14 2024-05-21T12:36:14.891 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 124.0k 114.8k 0.0k 1.2M 1.1M 0.0k nvme8n15 2024-05-21T12:36:14.891 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 154.0k 173.2k 0.0k 1.5M 1.7M 0.0k nvme8n16 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 166.8k 125.2k 0.0k 1.6M 1.2M 0.0k nvme8n17 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 148.0k 156.0k 0.0k 1.4M 1.5M 0.0k nvme8n18 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 150.4k 185.6k 0.0k 1.5M 1.8M 0.0k nvme8n19 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 164.8k 138.0k 0.0k 1.6M 1.3M 0.0k nvme8n2 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 166.8k 159.2k 0.0k 1.6M 1.6M 0.0k nvme8n20 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 170.4k 142.8k 0.0k 1.7M 1.4M 0.0k nvme8n3 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 149.6k 132.4k 0.0k 1.5M 1.3M 0.0k nvme8n4 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 133.2k 148.0k 0.0k 1.3M 1.4M 0.0k nvme8n5 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 140.8k 161.6k 0.0k 1.4M 1.6M 0.0k nvme8n6 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 152.4k 127.2k 0.0k 1.5M 1.2M 0.0k nvme8n7 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 168.0k 140.4k 0.0k 1.6M 1.4M 0.0k nvme8n8 2024-05-21T12:36:14.892 INFO:tasks.workunit.client.3.smithi137.stdout: 10.70 127.6k 123.2k 0.0k 1.2M 1.2M 0.0k nvme8n9 2024-05-21T12:36:14.893 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:14.893 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.893 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:36:14.893 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.838841] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:36:14.893 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 2024-05-21T12:36:14.893 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:14.893 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.894 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:36:14.894 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.858885] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:36:14.894 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 2024-05-21T12:36:14.894 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:14.894 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:14.894 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:36:14.894 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.879944] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:36:14.894 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b 2024-05-21T12:36:15.134 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:15.134 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.134 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:36:15.134 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.900845] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.920942] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.941709] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:36:15.135 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.961342] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:14.982106] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:14 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:14] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:36:15.136 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:15.002550] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:36:15.137 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc 2024-05-21T12:36:15.137 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:36:15.137 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.137 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:36:15.137 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:15.022812] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:36:15.137 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b 2024-05-21T12:36:15.137 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:15.044407] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:15.064891] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:36:15.138 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:15.085595] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:15.122208] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.72 port 4420 *** 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: None 2024-05-21T12:36:15.139 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: None 2024-05-21T12:36:15.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-mon[33912]: pgmap v655: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 636 op/s 2024-05-21T12:36:15.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:15 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:15] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: None 2024-05-21T12:36:15.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-mon[33665]: pgmap v655: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 636 op/s 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:36:15] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:36:15.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:36:15.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:36:15.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:36:15.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:36:15.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:36:15.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:36:15.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:36:15.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:59246 2024-05-21T12:36:15.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:15 smithi045 ceph-mon[30165]: pgmap v655: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 636 op/s 2024-05-21T12:36:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:17 smithi072 ceph-mon[33912]: pgmap v656: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 749 op/s 2024-05-21T12:36:17.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:17 smithi092 ceph-mon[33665]: pgmap v656: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 749 op/s 2024-05-21T12:36:17.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:17 smithi045 ceph-mon[30165]: pgmap v656: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 8.2 MiB/s rd, 8.0 MiB/s wr, 749 op/s 2024-05-21T12:36:18.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:36:18] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:36:18.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 1 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: OPTIMIZED 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 1 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: OPTIMIZED 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:36:18.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 1 2024-05-21T12:36:18.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: OPTIMIZED 2024-05-21T12:36:18.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:36:18.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: ], , client address: IPv4 172.21.15.45:40836 2024-05-21T12:36:18.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:18.038555] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000279 seconds 2024-05-21T12:36:19.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:19 smithi045 ceph-mon[30165]: pgmap v657: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 9.1 MiB/s rd, 8.5 MiB/s wr, 940 op/s 2024-05-21T12:36:19.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-mon[33912]: pgmap v657: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 9.1 MiB/s rd, 8.5 MiB/s wr, 940 op/s 2024-05-21T12:36:19.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:36:19] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:36:19.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:19.415 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 2 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 2 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 2 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:36:19.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:60184 2024-05-21T12:36:19.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:19.042570] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_2_0 elapsed time: 0.000405 seconds 2024-05-21T12:36:19.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:19 smithi092 ceph-mon[33665]: pgmap v657: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 9.1 MiB/s rd, 8.5 MiB/s wr, 940 op/s 2024-05-21T12:36:21.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:21 smithi092 ceph-mon[33665]: pgmap v658: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.8 MiB/s rd, 7.0 MiB/s wr, 825 op/s 2024-05-21T12:36:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:21 smithi045 ceph-mon[30165]: pgmap v658: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.8 MiB/s rd, 7.0 MiB/s wr, 825 op/s 2024-05-21T12:36:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:21 smithi072 ceph-mon[33912]: pgmap v658: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.8 MiB/s rd, 7.0 MiB/s wr, 825 op/s 2024-05-21T12:36:23.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:23 smithi045 ceph-mon[30165]: pgmap v659: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 794 op/s 2024-05-21T12:36:23.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:23 smithi092 ceph-mon[33665]: pgmap v659: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 794 op/s 2024-05-21T12:36:23.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:23 smithi072 ceph-mon[33912]: pgmap v659: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 794 op/s 2024-05-21T12:36:24.866 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:36:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 85.2k 64.4k 0.0k 852.0k 644.0k 0.0k nvme2n1 2024-05-21T12:36:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 92.0k 67.2k 0.0k 920.0k 672.0k 0.0k nvme2n10 2024-05-21T12:36:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 6.70 115.6k 64.8k 0.0k 1.1M 648.0k 0.0k nvme2n11 2024-05-21T12:36:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 5.80 104.8k 44.8k 0.0k 1.0M 448.0k 0.0k nvme2n12 2024-05-21T12:36:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 6.50 74.0k 52.4k 0.0k 740.0k 524.0k 0.0k nvme2n13 2024-05-21T12:36:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 5.90 137.6k 41.2k 0.0k 1.3M 412.0k 0.0k nvme2n14 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.50 112.0k 89.6k 0.0k 1.1M 896.0k 0.0k nvme2n15 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 98.4k 55.2k 0.0k 984.0k 552.0k 0.0k nvme2n16 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 118.0k 62.4k 0.0k 1.2M 624.0k 0.0k nvme2n17 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.80 102.0k 82.4k 0.0k 1020.0k 824.0k 0.0k nvme2n18 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 120.0k 60.0k 0.0k 1.2M 600.0k 0.0k nvme2n19 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 116.8k 50.4k 0.0k 1.1M 504.0k 0.0k nvme2n2 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.00 119.2k 78.4k 0.0k 1.2M 784.0k 0.0k nvme2n20 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 103.2k 46.8k 0.0k 1.0M 468.0k 0.0k nvme2n3 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 103.2k 63.2k 0.0k 1.0M 632.0k 0.0k nvme2n4 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 97.6k 48.4k 0.0k 976.0k 484.0k 0.0k nvme2n5 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 5.50 106.8k 64.4k 0.0k 1.0M 644.0k 0.0k nvme2n6 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.10 114.0k 49.6k 0.0k 1.1M 496.0k 0.0k nvme2n7 2024-05-21T12:36:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 80.4k 80.0k 0.0k 804.0k 800.0k 0.0k nvme2n8 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.50 94.8k 67.6k 0.0k 948.0k 676.0k 0.0k nvme2n9 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.00 96.8k 79.6k 0.0k 968.0k 796.0k 0.0k nvme5n1 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.60 96.8k 54.4k 0.0k 968.0k 544.0k 0.0k nvme5n10 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 137.6k 39.2k 0.0k 1.3M 392.0k 0.0k nvme5n11 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 115.2k 46.4k 0.0k 1.1M 464.0k 0.0k nvme5n12 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 102.4k 44.4k 0.0k 1.0M 444.0k 0.0k nvme5n13 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 5.70 102.4k 60.4k 0.0k 1.0M 604.0k 0.0k nvme5n14 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.70 105.2k 74.0k 0.0k 1.0M 740.0k 0.0k nvme5n15 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.60 119.2k 50.0k 0.0k 1.2M 500.0k 0.0k nvme5n16 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 5.70 94.8k 64.8k 0.0k 948.0k 648.0k 0.0k nvme5n17 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.60 110.0k 64.8k 0.0k 1.1M 648.0k 0.0k nvme5n18 2024-05-21T12:36:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.60 121.2k 64.4k 0.0k 1.2M 644.0k 0.0k nvme5n19 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.70 98.0k 76.0k 0.0k 980.0k 760.0k 0.0k nvme5n2 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 106.0k 74.0k 0.0k 1.0M 740.0k 0.0k nvme5n20 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.50 112.0k 65.6k 0.0k 1.1M 656.0k 0.0k nvme5n3 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.60 94.8k 73.6k 0.0k 948.0k 736.0k 0.0k nvme5n4 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 127.2k 59.2k 0.0k 1.2M 592.0k 0.0k nvme5n5 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 5.90 107.6k 57.2k 0.0k 1.1M 572.0k 0.0k nvme5n6 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.60 116.0k 62.8k 0.0k 1.1M 628.0k 0.0k nvme5n7 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 118.4k 56.0k 0.0k 1.2M 560.0k 0.0k nvme5n8 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.00 112.0k 56.0k 0.0k 1.1M 560.0k 0.0k nvme5n9 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.10 122.4k 61.6k 0.0k 1.2M 616.0k 0.0k nvme8n1 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 100.4k 66.4k 0.0k 1004.0k 664.0k 0.0k nvme8n10 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 106.0k 51.2k 0.0k 1.0M 512.0k 0.0k nvme8n11 2024-05-21T12:36:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 107.6k 69.2k 0.0k 1.1M 692.0k 0.0k nvme8n12 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 97.6k 75.6k 0.0k 976.0k 756.0k 0.0k nvme8n13 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.10 122.0k 63.2k 0.0k 1.2M 632.0k 0.0k nvme8n14 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.10 129.6k 44.0k 0.0k 1.3M 440.0k 0.0k nvme8n15 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.90 104.4k 50.0k 0.0k 1.0M 500.0k 0.0k nvme8n16 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 92.8k 74.8k 0.0k 928.0k 748.0k 0.0k nvme8n17 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 105.6k 66.8k 0.0k 1.0M 668.0k 0.0k nvme8n18 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.80 107.2k 85.2k 0.0k 1.0M 852.0k 0.0k nvme8n19 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.10 100.8k 69.2k 0.0k 1008.0k 692.0k 0.0k nvme8n2 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 112.4k 62.4k 0.0k 1.1M 624.0k 0.0k nvme8n20 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.60 91.6k 63.2k 0.0k 916.0k 632.0k 0.0k nvme8n3 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.50 111.2k 54.8k 0.0k 1.1M 548.0k 0.0k nvme8n4 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 80.4k 58.8k 0.0k 804.0k 588.0k 0.0k nvme8n5 2024-05-21T12:36:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 5.60 92.8k 74.8k 0.0k 928.0k 748.0k 0.0k nvme8n6 2024-05-21T12:36:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 6.20 140.0k 58.0k 0.0k 1.4M 580.0k 0.0k nvme8n7 2024-05-21T12:36:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 6.50 100.0k 34.8k 0.0k 1000.0k 348.0k 0.0k nvme8n8 2024-05-21T12:36:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 5.70 100.4k 71.6k 0.0k 1004.0k 716.0k 0.0k nvme8n9 2024-05-21T12:36:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:25.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:25 smithi092 ceph-mon[33665]: pgmap v660: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.7 MiB/s wr, 752 op/s 2024-05-21T12:36:25.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:25 smithi045 ceph-mon[30165]: pgmap v660: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.7 MiB/s wr, 752 op/s 2024-05-21T12:36:25.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:25 smithi072 ceph-mon[33912]: pgmap v660: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.7 MiB/s wr, 752 op/s 2024-05-21T12:36:27.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:27 smithi092 ceph-mon[33665]: pgmap v661: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 759 op/s 2024-05-21T12:36:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:27 smithi045 ceph-mon[30165]: pgmap v661: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 759 op/s 2024-05-21T12:36:27.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:27 smithi072 ceph-mon[33912]: pgmap v661: 33 pgs: 33 active+clean; 1.0 GiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 759 op/s 2024-05-21T12:36:29.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:29 smithi092 ceph-mon[33665]: pgmap v662: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 734 op/s 2024-05-21T12:36:29.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:29 smithi045 ceph-mon[30165]: pgmap v662: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 734 op/s 2024-05-21T12:36:29.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:29 smithi072 ceph-mon[33912]: pgmap v662: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 734 op/s 2024-05-21T12:36:31.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:31 smithi092 ceph-mon[33665]: pgmap v663: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 544 op/s 2024-05-21T12:36:31.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:31 smithi045 ceph-mon[30165]: pgmap v663: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 544 op/s 2024-05-21T12:36:31.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:31 smithi072 ceph-mon[33912]: pgmap v663: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 544 op/s 2024-05-21T12:36:33.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:33 smithi092 ceph-mon[33665]: pgmap v664: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-21T12:36:33.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:33 smithi045 ceph-mon[30165]: pgmap v664: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-21T12:36:33.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:33 smithi072 ceph-mon[33912]: pgmap v664: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-21T12:36:34.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:36:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 137.6k 131.6k 0.0k 1.3M 1.3M 0.0k nvme2n1 2024-05-21T12:36:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 123.6k 134.0k 0.0k 1.2M 1.3M 0.0k nvme2n10 2024-05-21T12:36:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 129.2k 157.6k 0.0k 1.3M 1.5M 0.0k nvme2n11 2024-05-21T12:36:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 126.8k 188.0k 0.0k 1.2M 1.8M 0.0k nvme2n12 2024-05-21T12:36:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 179.6k 98.4k 0.0k 1.8M 984.0k 0.0k nvme2n13 2024-05-21T12:36:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 157.6k 143.6k 0.0k 1.5M 1.4M 0.0k nvme2n14 2024-05-21T12:36:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 144.4k 133.2k 0.0k 1.4M 1.3M 0.0k nvme2n15 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 139.6k 172.0k 0.0k 1.4M 1.7M 0.0k nvme2n16 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 154.8k 157.6k 0.0k 1.5M 1.5M 0.0k nvme2n17 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 140.8k 135.2k 0.0k 1.4M 1.3M 0.0k nvme2n18 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 187.6k 111.6k 0.0k 1.8M 1.1M 0.0k nvme2n19 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 127.2k 149.6k 0.0k 1.2M 1.5M 0.0k nvme2n2 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 131.2k 142.4k 0.0k 1.3M 1.4M 0.0k nvme2n20 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 143.6k 148.0k 0.0k 1.4M 1.4M 0.0k nvme2n3 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 160.4k 142.0k 0.0k 1.6M 1.4M 0.0k nvme2n4 2024-05-21T12:36:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 135.6k 132.8k 0.0k 1.3M 1.3M 0.0k nvme2n5 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 135.6k 132.0k 0.0k 1.3M 1.3M 0.0k nvme2n6 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 144.8k 113.2k 0.0k 1.4M 1.1M 0.0k nvme2n7 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 126.8k 144.4k 0.0k 1.2M 1.4M 0.0k nvme2n8 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 138.8k 162.8k 0.0k 1.4M 1.6M 0.0k nvme2n9 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 156.8k 131.6k 0.0k 1.5M 1.3M 0.0k nvme5n1 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 184.0k 131.6k 0.0k 1.8M 1.3M 0.0k nvme5n10 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 128.0k 130.8k 0.0k 1.2M 1.3M 0.0k nvme5n11 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 176.0k 132.8k 0.0k 1.7M 1.3M 0.0k nvme5n12 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 145.2k 158.4k 0.0k 1.4M 1.5M 0.0k nvme5n13 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 174.4k 140.4k 0.0k 1.7M 1.4M 0.0k nvme5n14 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 164.4k 140.4k 0.0k 1.6M 1.4M 0.0k nvme5n15 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 140.4k 165.2k 0.0k 1.4M 1.6M 0.0k nvme5n16 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 126.8k 131.6k 0.0k 1.2M 1.3M 0.0k nvme5n17 2024-05-21T12:36:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 149.6k 140.0k 0.0k 1.5M 1.4M 0.0k nvme5n18 2024-05-21T12:36:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 129.2k 185.2k 0.0k 1.3M 1.8M 0.0k nvme5n19 2024-05-21T12:36:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 175.6k 114.8k 0.0k 1.7M 1.1M 0.0k nvme5n2 2024-05-21T12:36:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 138.8k 170.4k 0.0k 1.4M 1.7M 0.0k nvme5n20 2024-05-21T12:36:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 187.6k 122.4k 0.0k 1.8M 1.2M 0.0k nvme5n3 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 141.6k 152.0k 0.0k 1.4M 1.5M 0.0k nvme5n4 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 162.8k 127.6k 0.0k 1.6M 1.2M 0.0k nvme5n5 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 168.8k 149.2k 0.0k 1.6M 1.5M 0.0k nvme5n6 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 126.8k 160.4k 0.0k 1.2M 1.6M 0.0k nvme5n7 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 169.2k 143.2k 0.0k 1.7M 1.4M 0.0k nvme5n8 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 123.6k 152.8k 0.0k 1.2M 1.5M 0.0k nvme5n9 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 185.2k 86.0k 0.0k 1.8M 860.0k 0.0k nvme8n1 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 123.2k 126.0k 0.0k 1.2M 1.2M 0.0k nvme8n10 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 146.8k 159.6k 0.0k 1.4M 1.6M 0.0k nvme8n11 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 146.4k 169.6k 0.0k 1.4M 1.7M 0.0k nvme8n12 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 136.0k 186.8k 0.0k 1.3M 1.8M 0.0k nvme8n13 2024-05-21T12:36:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 130.4k 134.4k 0.0k 1.3M 1.3M 0.0k nvme8n14 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 136.4k 136.4k 0.0k 1.3M 1.3M 0.0k nvme8n15 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 130.8k 123.6k 0.0k 1.3M 1.2M 0.0k nvme8n16 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 113.6k 190.4k 0.0k 1.1M 1.9M 0.0k nvme8n17 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 148.8k 124.4k 0.0k 1.5M 1.2M 0.0k nvme8n18 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 154.8k 113.2k 0.0k 1.5M 1.1M 0.0k nvme8n19 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 111.2k 182.4k 0.0k 1.1M 1.8M 0.0k nvme8n2 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 162.4k 148.4k 0.0k 1.6M 1.4M 0.0k nvme8n20 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 170.0k 128.0k 0.0k 1.7M 1.2M 0.0k nvme8n3 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 189.2k 98.0k 0.0k 1.8M 980.0k 0.0k nvme8n4 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 132.0k 154.4k 0.0k 1.3M 1.5M 0.0k nvme8n5 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 168.4k 128.4k 0.0k 1.6M 1.3M 0.0k nvme8n6 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 176.4k 138.8k 0.0k 1.7M 1.4M 0.0k nvme8n7 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 171.2k 127.2k 0.0k 1.7M 1.2M 0.0k nvme8n8 2024-05-21T12:36:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 172.4k 164.4k 0.0k 1.7M 1.6M 0.0k nvme8n9 2024-05-21T12:36:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:35.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:35 smithi092 ceph-mon[33665]: pgmap v665: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-05-21T12:36:35.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:35 smithi045 ceph-mon[30165]: pgmap v665: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-05-21T12:36:35.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:35 smithi072 ceph-mon[33912]: pgmap v665: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.5 MiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-05-21T12:36:37.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:37 smithi092 ceph-mon[33665]: pgmap v666: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.6 MiB/s rd, 7.9 MiB/s wr, 613 op/s 2024-05-21T12:36:37.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:37 smithi045 ceph-mon[30165]: pgmap v666: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.6 MiB/s rd, 7.9 MiB/s wr, 613 op/s 2024-05-21T12:36:37.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:37 smithi072 ceph-mon[33912]: pgmap v666: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.6 MiB/s rd, 7.9 MiB/s wr, 613 op/s 2024-05-21T12:36:39.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:39 smithi092 ceph-mon[33665]: pgmap v667: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 682 op/s 2024-05-21T12:36:39.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:39 smithi045 ceph-mon[30165]: pgmap v667: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 682 op/s 2024-05-21T12:36:39.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:39 smithi072 ceph-mon[33912]: pgmap v667: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 9.4 MiB/s rd, 9.1 MiB/s wr, 682 op/s 2024-05-21T12:36:41.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:41 smithi092 ceph-mon[33665]: pgmap v668: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.0 MiB/s wr, 616 op/s 2024-05-21T12:36:41.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:41 smithi045 ceph-mon[30165]: pgmap v668: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.0 MiB/s wr, 616 op/s 2024-05-21T12:36:41.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:41 smithi072 ceph-mon[33912]: pgmap v668: 33 pgs: 33 active+clean; 1.0 GiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.0 MiB/s wr, 616 op/s 2024-05-21T12:36:43.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:43 smithi092 ceph-mon[33665]: pgmap v669: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 637 op/s 2024-05-21T12:36:43.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:43 smithi045 ceph-mon[30165]: pgmap v669: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 637 op/s 2024-05-21T12:36:43.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:43 smithi072 ceph-mon[33912]: pgmap v669: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 637 op/s 2024-05-21T12:36:44.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 136.4k 122.8k 0.0k 1.3M 1.2M 0.0k nvme2n1 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 77.2k 163.2k 0.0k 772.0k 1.6M 0.0k nvme2n10 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 127.6k 162.4k 0.0k 1.2M 1.6M 0.0k nvme2n11 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 148.4k 112.0k 0.0k 1.4M 1.1M 0.0k nvme2n12 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 129.2k 129.2k 0.0k 1.3M 1.3M 0.0k nvme2n13 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 141.6k 136.4k 0.0k 1.4M 1.3M 0.0k nvme2n14 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 156.4k 119.2k 0.0k 1.5M 1.2M 0.0k nvme2n15 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 141.2k 135.6k 0.0k 1.4M 1.3M 0.0k nvme2n16 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 115.6k 166.0k 0.0k 1.1M 1.6M 0.0k nvme2n17 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 132.4k 124.8k 0.0k 1.3M 1.2M 0.0k nvme2n18 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 186.4k 120.0k 0.0k 1.8M 1.2M 0.0k nvme2n19 2024-05-21T12:36:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 122.8k 155.2k 0.0k 1.2M 1.5M 0.0k nvme2n2 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 133.6k 111.6k 0.0k 1.3M 1.1M 0.0k nvme2n20 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 128.8k 156.4k 0.0k 1.3M 1.5M 0.0k nvme2n3 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 146.0k 124.8k 0.0k 1.4M 1.2M 0.0k nvme2n4 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 115.6k 146.8k 0.0k 1.1M 1.4M 0.0k nvme2n5 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 114.8k 104.8k 0.0k 1.1M 1.0M 0.0k nvme2n6 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 145.6k 132.4k 0.0k 1.4M 1.3M 0.0k nvme2n7 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 129.6k 127.6k 0.0k 1.3M 1.2M 0.0k nvme2n8 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 118.8k 144.4k 0.0k 1.2M 1.4M 0.0k nvme2n9 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 128.0k 115.6k 0.0k 1.2M 1.1M 0.0k nvme5n1 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 138.4k 144.8k 0.0k 1.4M 1.4M 0.0k nvme5n10 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 144.0k 136.8k 0.0k 1.4M 1.3M 0.0k nvme5n11 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 121.2k 139.2k 0.0k 1.2M 1.4M 0.0k nvme5n12 2024-05-21T12:36:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 113.2k 126.4k 0.0k 1.1M 1.2M 0.0k nvme5n13 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 150.4k 137.6k 0.0k 1.5M 1.3M 0.0k nvme5n14 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 120.8k 152.8k 0.0k 1.2M 1.5M 0.0k nvme5n15 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 124.0k 140.0k 0.0k 1.2M 1.4M 0.0k nvme5n16 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 108.0k 131.2k 0.0k 1.1M 1.3M 0.0k nvme5n17 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 82.4k 155.2k 0.0k 824.0k 1.5M 0.0k nvme5n18 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 146.8k 129.6k 0.0k 1.4M 1.3M 0.0k nvme5n19 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 141.6k 131.2k 0.0k 1.4M 1.3M 0.0k nvme5n2 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 117.2k 152.8k 0.0k 1.1M 1.5M 0.0k nvme5n20 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 150.0k 136.4k 0.0k 1.5M 1.3M 0.0k nvme5n3 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 136.4k 148.8k 0.0k 1.3M 1.5M 0.0k nvme5n4 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 123.2k 156.8k 0.0k 1.2M 1.5M 0.0k nvme5n5 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 145.2k 134.0k 0.0k 1.4M 1.3M 0.0k nvme5n6 2024-05-21T12:36:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 132.0k 122.8k 0.0k 1.3M 1.2M 0.0k nvme5n7 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 136.0k 125.6k 0.0k 1.3M 1.2M 0.0k nvme5n8 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 144.4k 120.0k 0.0k 1.4M 1.2M 0.0k nvme5n9 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 163.6k 105.2k 0.0k 1.6M 1.0M 0.0k nvme8n1 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 156.0k 107.2k 0.0k 1.5M 1.0M 0.0k nvme8n10 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 127.6k 116.4k 0.0k 1.2M 1.1M 0.0k nvme8n11 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 132.0k 160.0k 0.0k 1.3M 1.6M 0.0k nvme8n12 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 153.2k 110.8k 0.0k 1.5M 1.1M 0.0k nvme8n13 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 126.8k 114.8k 0.0k 1.2M 1.1M 0.0k nvme8n14 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 140.4k 139.6k 0.0k 1.4M 1.4M 0.0k nvme8n15 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 139.2k 100.4k 0.0k 1.4M 1004.0k 0.0k nvme8n16 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 131.6k 128.4k 0.0k 1.3M 1.3M 0.0k nvme8n17 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 135.6k 144.8k 0.0k 1.3M 1.4M 0.0k nvme8n18 2024-05-21T12:36:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.30 118.4k 140.0k 0.0k 1.2M 1.4M 0.0k nvme8n19 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 119.6k 134.8k 0.0k 1.2M 1.3M 0.0k nvme8n2 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 119.2k 140.4k 0.0k 1.2M 1.4M 0.0k nvme8n20 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 115.6k 128.0k 0.0k 1.1M 1.2M 0.0k nvme8n3 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 125.6k 152.0k 0.0k 1.2M 1.5M 0.0k nvme8n4 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 103.2k 160.8k 0.0k 1.0M 1.6M 0.0k nvme8n5 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 164.4k 114.8k 0.0k 1.6M 1.1M 0.0k nvme8n6 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 148.0k 152.0k 0.0k 1.4M 1.5M 0.0k nvme8n7 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 112.8k 148.4k 0.0k 1.1M 1.4M 0.0k nvme8n8 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.40 129.6k 136.8k 0.0k 1.3M 1.3M 0.0k nvme8n9 2024-05-21T12:36:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:45.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:45 smithi092 ceph-mon[33665]: pgmap v670: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 597 op/s 2024-05-21T12:36:45.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:45 smithi045 ceph-mon[30165]: pgmap v670: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 597 op/s 2024-05-21T12:36:45.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:45 smithi072 ceph-mon[33912]: pgmap v670: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 597 op/s 2024-05-21T12:36:47.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:47 smithi092 ceph-mon[33665]: pgmap v671: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 610 op/s 2024-05-21T12:36:47.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:47 smithi045 ceph-mon[30165]: pgmap v671: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 610 op/s 2024-05-21T12:36:47.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:47 smithi072 ceph-mon[33912]: pgmap v671: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 610 op/s 2024-05-21T12:36:49.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:49 smithi092 ceph-mon[33665]: pgmap v672: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.5 MiB/s wr, 682 op/s 2024-05-21T12:36:49.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:49 smithi045 ceph-mon[30165]: pgmap v672: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.5 MiB/s wr, 682 op/s 2024-05-21T12:36:49.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:49 smithi072 ceph-mon[33912]: pgmap v672: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 8.3 MiB/s rd, 8.5 MiB/s wr, 682 op/s 2024-05-21T12:36:51.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:51 smithi092 ceph-mon[33665]: pgmap v673: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 623 op/s 2024-05-21T12:36:51.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:51 smithi045 ceph-mon[30165]: pgmap v673: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 623 op/s 2024-05-21T12:36:51.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:51 smithi072 ceph-mon[33912]: pgmap v673: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 623 op/s 2024-05-21T12:36:53.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:53 smithi092 ceph-mon[33665]: pgmap v674: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 646 op/s 2024-05-21T12:36:53.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:53 smithi045 ceph-mon[30165]: pgmap v674: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 646 op/s 2024-05-21T12:36:53.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:53 smithi072 ceph-mon[33912]: pgmap v674: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 646 op/s 2024-05-21T12:36:54.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:36:54.874 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 137.6k 97.6k 0.0k 1.3M 976.0k 0.0k nvme2n1 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 107.2k 147.2k 0.0k 1.0M 1.4M 0.0k nvme2n10 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 114.0k 100.0k 0.0k 1.1M 1000.0k 0.0k nvme2n11 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 124.8k 119.6k 0.0k 1.2M 1.2M 0.0k nvme2n12 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 122.0k 129.2k 0.0k 1.2M 1.3M 0.0k nvme2n13 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 143.2k 121.6k 0.0k 1.4M 1.2M 0.0k nvme2n14 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 116.0k 121.6k 0.0k 1.1M 1.2M 0.0k nvme2n15 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 133.2k 127.6k 0.0k 1.3M 1.2M 0.0k nvme2n16 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 135.2k 133.2k 0.0k 1.3M 1.3M 0.0k nvme2n17 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 154.8k 121.6k 0.0k 1.5M 1.2M 0.0k nvme2n18 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 106.8k 135.6k 0.0k 1.0M 1.3M 0.0k nvme2n19 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 106.0k 128.8k 0.0k 1.0M 1.3M 0.0k nvme2n2 2024-05-21T12:36:54.875 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 149.6k 106.4k 0.0k 1.5M 1.0M 0.0k nvme2n20 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 127.2k 128.0k 0.0k 1.2M 1.2M 0.0k nvme2n3 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 111.6k 102.4k 0.0k 1.1M 1.0M 0.0k nvme2n4 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 128.4k 137.2k 0.0k 1.3M 1.3M 0.0k nvme2n5 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 126.8k 100.8k 0.0k 1.2M 1008.0k 0.0k nvme2n6 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 102.0k 139.2k 0.0k 1020.0k 1.4M 0.0k nvme2n7 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 105.6k 117.2k 0.0k 1.0M 1.1M 0.0k nvme2n8 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 118.8k 96.8k 0.0k 1.2M 968.0k 0.0k nvme2n9 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 122.4k 134.4k 0.0k 1.2M 1.3M 0.0k nvme5n1 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 126.4k 152.0k 0.0k 1.2M 1.5M 0.0k nvme5n10 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 100.8k 136.8k 0.0k 1008.0k 1.3M 0.0k nvme5n11 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 136.4k 105.2k 0.0k 1.3M 1.0M 0.0k nvme5n12 2024-05-21T12:36:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 110.4k 139.2k 0.0k 1.1M 1.4M 0.0k nvme5n13 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 103.2k 143.2k 0.0k 1.0M 1.4M 0.0k nvme5n14 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 130.8k 140.0k 0.0k 1.3M 1.4M 0.0k nvme5n15 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 117.6k 87.6k 0.0k 1.1M 876.0k 0.0k nvme5n16 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 108.8k 128.4k 0.0k 1.1M 1.3M 0.0k nvme5n17 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 98.0k 137.2k 0.0k 980.0k 1.3M 0.0k nvme5n18 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 141.2k 122.0k 0.0k 1.4M 1.2M 0.0k nvme5n19 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 106.8k 103.6k 0.0k 1.0M 1.0M 0.0k nvme5n2 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 148.4k 112.4k 0.0k 1.4M 1.1M 0.0k nvme5n20 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 106.4k 141.6k 0.0k 1.0M 1.4M 0.0k nvme5n3 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 77.6k 168.0k 0.0k 776.0k 1.6M 0.0k nvme5n4 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 92.0k 117.2k 0.0k 920.0k 1.1M 0.0k nvme5n5 2024-05-21T12:36:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 101.6k 119.6k 0.0k 1016.0k 1.2M 0.0k nvme5n6 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 113.2k 96.8k 0.0k 1.1M 968.0k 0.0k nvme5n7 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 124.0k 108.4k 0.0k 1.2M 1.1M 0.0k nvme5n8 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 116.8k 118.4k 0.0k 1.1M 1.2M 0.0k nvme5n9 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 98.4k 143.6k 0.0k 984.0k 1.4M 0.0k nvme8n1 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 135.6k 106.8k 0.0k 1.3M 1.0M 0.0k nvme8n10 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 125.2k 124.8k 0.0k 1.2M 1.2M 0.0k nvme8n11 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 96.8k 150.4k 0.0k 968.0k 1.5M 0.0k nvme8n12 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 103.2k 118.4k 0.0k 1.0M 1.2M 0.0k nvme8n13 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 98.0k 142.8k 0.0k 980.0k 1.4M 0.0k nvme8n14 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 124.0k 130.4k 0.0k 1.2M 1.3M 0.0k nvme8n15 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 123.6k 130.4k 0.0k 1.2M 1.3M 0.0k nvme8n16 2024-05-21T12:36:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 110.8k 146.8k 0.0k 1.1M 1.4M 0.0k nvme8n17 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 109.2k 135.6k 0.0k 1.1M 1.3M 0.0k nvme8n18 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 116.4k 111.6k 0.0k 1.1M 1.1M 0.0k nvme8n19 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 139.2k 129.6k 0.0k 1.4M 1.3M 0.0k nvme8n2 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 120.4k 105.6k 0.0k 1.2M 1.0M 0.0k nvme8n20 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 139.6k 114.4k 0.0k 1.4M 1.1M 0.0k nvme8n3 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 133.2k 133.6k 0.0k 1.3M 1.3M 0.0k nvme8n4 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 104.0k 136.8k 0.0k 1.0M 1.3M 0.0k nvme8n5 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 123.2k 118.8k 0.0k 1.2M 1.2M 0.0k nvme8n6 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 135.6k 116.0k 0.0k 1.3M 1.1M 0.0k nvme8n7 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 111.2k 107.2k 0.0k 1.1M 1.0M 0.0k nvme8n8 2024-05-21T12:36:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 134.4k 141.6k 0.0k 1.3M 1.4M 0.0k nvme8n9 2024-05-21T12:36:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:36:55.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:55 smithi092 ceph-mon[33665]: pgmap v675: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-05-21T12:36:55.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:55 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:36:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:55 smithi045 ceph-mon[30165]: pgmap v675: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-05-21T12:36:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:55 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:36:55.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:55 smithi072 ceph-mon[33912]: pgmap v675: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-05-21T12:36:55.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:55 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:36:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:56 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:36:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:56 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:36:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:56 smithi092 ceph-mon[33665]: pgmap v676: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-05-21T12:36:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:56 smithi092 ceph-mon[33665]: pgmap v677: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 689 op/s 2024-05-21T12:36:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:56 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:36:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:56 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:36:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:56 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:36:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:56 smithi045 ceph-mon[30165]: pgmap v676: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-05-21T12:36:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:56 smithi045 ceph-mon[30165]: pgmap v677: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 689 op/s 2024-05-21T12:36:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:56 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:36:56.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:56 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:36:56.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:56 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:36:56.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:56 smithi072 ceph-mon[33912]: pgmap v676: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-05-21T12:36:56.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:56 smithi072 ceph-mon[33912]: pgmap v677: 33 pgs: 33 active+clean; 1.0 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 7.5 MiB/s rd, 8.0 MiB/s wr, 689 op/s 2024-05-21T12:36:56.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:56 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:36:58.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:36:58 smithi072 ceph-mon[33912]: pgmap v678: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 630 op/s 2024-05-21T12:36:58.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:36:58 smithi092 ceph-mon[33665]: pgmap v678: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 630 op/s 2024-05-21T12:36:58.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:36:58 smithi045 ceph-mon[30165]: pgmap v678: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 630 op/s 2024-05-21T12:37:00.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:00 smithi072 ceph-mon[33912]: pgmap v679: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 595 op/s 2024-05-21T12:37:00.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:00 smithi092 ceph-mon[33665]: pgmap v679: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 595 op/s 2024-05-21T12:37:00.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:00 smithi045 ceph-mon[30165]: pgmap v679: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.6 MiB/s wr, 595 op/s 2024-05-21T12:37:02.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:02 smithi072 ceph-mon[33912]: pgmap v680: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-05-21T12:37:02.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:02 smithi092 ceph-mon[33665]: pgmap v680: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-05-21T12:37:02.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:02 smithi045 ceph-mon[30165]: pgmap v680: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-05-21T12:37:04.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:04 smithi072 ceph-mon[33912]: pgmap v681: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-05-21T12:37:04.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:04 smithi092 ceph-mon[33665]: pgmap v681: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-05-21T12:37:04.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:04 smithi045 ceph-mon[30165]: pgmap v681: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-05-21T12:37:04.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:37:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 108.0k 90.4k 0.0k 1.1M 904.0k 0.0k nvme2n1 2024-05-21T12:37:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 105.6k 95.2k 0.0k 1.0M 952.0k 0.0k nvme2n10 2024-05-21T12:37:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 92.8k 130.4k 0.0k 928.0k 1.3M 0.0k nvme2n11 2024-05-21T12:37:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 114.8k 86.8k 0.0k 1.1M 868.0k 0.0k nvme2n12 2024-05-21T12:37:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 104.8k 97.6k 0.0k 1.0M 976.0k 0.0k nvme2n13 2024-05-21T12:37:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 131.2k 93.6k 0.0k 1.3M 936.0k 0.0k nvme2n14 2024-05-21T12:37:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 112.8k 91.2k 0.0k 1.1M 912.0k 0.0k nvme2n15 2024-05-21T12:37:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 99.2k 125.2k 0.0k 992.0k 1.2M 0.0k nvme2n16 2024-05-21T12:37:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 148.0k 94.4k 0.0k 1.4M 944.0k 0.0k nvme2n17 2024-05-21T12:37:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 98.8k 108.4k 0.0k 988.0k 1.1M 0.0k nvme2n18 2024-05-21T12:37:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 80.8k 116.8k 0.0k 808.0k 1.1M 0.0k nvme2n19 2024-05-21T12:37:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 120.0k 116.0k 0.0k 1.2M 1.1M 0.0k nvme2n2 2024-05-21T12:37:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 76.8k 110.8k 0.0k 768.0k 1.1M 0.0k nvme2n20 2024-05-21T12:37:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 95.6k 116.4k 0.0k 956.0k 1.1M 0.0k nvme2n3 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 124.0k 92.8k 0.0k 1.2M 928.0k 0.0k nvme2n4 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 114.8k 111.2k 0.0k 1.1M 1.1M 0.0k nvme2n5 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 116.8k 104.0k 0.0k 1.1M 1.0M 0.0k nvme2n6 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 95.6k 113.6k 0.0k 956.0k 1.1M 0.0k nvme2n7 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 84.0k 108.4k 0.0k 840.0k 1.1M 0.0k nvme2n8 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 124.8k 96.4k 0.0k 1.2M 964.0k 0.0k nvme2n9 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 96.0k 107.6k 0.0k 960.0k 1.1M 0.0k nvme5n1 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 93.2k 132.4k 0.0k 932.0k 1.3M 0.0k nvme5n10 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 110.8k 91.2k 0.0k 1.1M 912.0k 0.0k nvme5n11 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 83.2k 108.0k 0.0k 832.0k 1.1M 0.0k nvme5n12 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 114.0k 84.8k 0.0k 1.1M 848.0k 0.0k nvme5n13 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 111.2k 100.8k 0.0k 1.1M 1008.0k 0.0k nvme5n14 2024-05-21T12:37:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 103.6k 144.4k 0.0k 1.0M 1.4M 0.0k nvme5n15 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 110.8k 92.8k 0.0k 1.1M 928.0k 0.0k nvme5n16 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 108.4k 81.2k 0.0k 1.1M 812.0k 0.0k nvme5n17 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 107.2k 90.0k 0.0k 1.0M 900.0k 0.0k nvme5n18 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 128.0k 110.0k 0.0k 1.2M 1.1M 0.0k nvme5n19 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 109.2k 100.8k 0.0k 1.1M 1008.0k 0.0k nvme5n2 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 107.2k 97.6k 0.0k 1.0M 976.0k 0.0k nvme5n20 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 101.2k 134.0k 0.0k 1012.0k 1.3M 0.0k nvme5n3 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 89.6k 117.2k 0.0k 896.0k 1.1M 0.0k nvme5n4 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 100.0k 80.0k 0.0k 1000.0k 800.0k 0.0k nvme5n5 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 112.0k 96.8k 0.0k 1.1M 968.0k 0.0k nvme5n6 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 85.2k 95.2k 0.0k 852.0k 952.0k 0.0k nvme5n7 2024-05-21T12:37:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 101.6k 119.6k 0.0k 1016.0k 1.2M 0.0k nvme5n8 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 88.0k 88.0k 0.0k 880.0k 880.0k 0.0k nvme5n9 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 80.4k 134.0k 0.0k 804.0k 1.3M 0.0k nvme8n1 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 109.6k 88.4k 0.0k 1.1M 884.0k 0.0k nvme8n10 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 104.4k 97.2k 0.0k 1.0M 972.0k 0.0k nvme8n11 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 100.4k 124.0k 0.0k 1004.0k 1.2M 0.0k nvme8n12 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 101.6k 99.6k 0.0k 1016.0k 996.0k 0.0k nvme8n13 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 91.6k 114.0k 0.0k 916.0k 1.1M 0.0k nvme8n14 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 87.2k 128.0k 0.0k 872.0k 1.2M 0.0k nvme8n15 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 88.8k 112.4k 0.0k 888.0k 1.1M 0.0k nvme8n16 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 118.8k 123.6k 0.0k 1.2M 1.2M 0.0k nvme8n17 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 92.4k 122.8k 0.0k 924.0k 1.2M 0.0k nvme8n18 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 92.0k 116.0k 0.0k 920.0k 1.1M 0.0k nvme8n19 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 92.4k 126.4k 0.0k 924.0k 1.2M 0.0k nvme8n2 2024-05-21T12:37:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 84.4k 114.0k 0.0k 844.0k 1.1M 0.0k nvme8n20 2024-05-21T12:37:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 85.6k 123.2k 0.0k 856.0k 1.2M 0.0k nvme8n3 2024-05-21T12:37:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.10 94.4k 102.4k 0.0k 944.0k 1.0M 0.0k nvme8n4 2024-05-21T12:37:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 121.2k 91.2k 0.0k 1.2M 912.0k 0.0k nvme8n5 2024-05-21T12:37:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 126.4k 87.6k 0.0k 1.2M 876.0k 0.0k nvme8n6 2024-05-21T12:37:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 127.2k 97.6k 0.0k 1.2M 976.0k 0.0k nvme8n7 2024-05-21T12:37:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 72.8k 110.8k 0.0k 728.0k 1.1M 0.0k nvme8n8 2024-05-21T12:37:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.00 95.6k 103.6k 0.0k 956.0k 1.0M 0.0k nvme8n9 2024-05-21T12:37:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:06.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:06 smithi072 ceph-mon[33912]: pgmap v682: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 618 op/s 2024-05-21T12:37:06.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:06 smithi092 ceph-mon[33665]: pgmap v682: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 618 op/s 2024-05-21T12:37:06.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:06 smithi045 ceph-mon[30165]: pgmap v682: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 618 op/s 2024-05-21T12:37:08.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:08 smithi072 ceph-mon[33912]: pgmap v683: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 626 op/s 2024-05-21T12:37:08.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:08 smithi092 ceph-mon[33665]: pgmap v683: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 626 op/s 2024-05-21T12:37:08.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:08 smithi045 ceph-mon[30165]: pgmap v683: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 626 op/s 2024-05-21T12:37:10.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:10 smithi072 ceph-mon[33912]: pgmap v684: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 585 op/s 2024-05-21T12:37:10.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:10 smithi092 ceph-mon[33665]: pgmap v684: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 585 op/s 2024-05-21T12:37:10.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:10 smithi045 ceph-mon[30165]: pgmap v684: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 585 op/s 2024-05-21T12:37:12.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:12 smithi072 ceph-mon[33912]: pgmap v685: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-05-21T12:37:12.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:12 smithi092 ceph-mon[33665]: pgmap v685: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-05-21T12:37:12.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:12 smithi045 ceph-mon[30165]: pgmap v685: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-05-21T12:37:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:14 smithi072 ceph-mon[33912]: pgmap v686: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 677 op/s 2024-05-21T12:37:14.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:14 smithi092 ceph-mon[33665]: pgmap v686: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 677 op/s 2024-05-21T12:37:14.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:14 smithi045 ceph-mon[30165]: pgmap v686: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 677 op/s 2024-05-21T12:37:14.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:37:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 103.2k 98.0k 0.0k 1.0M 980.0k 0.0k nvme2n1 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 98.4k 114.0k 0.0k 984.0k 1.1M 0.0k nvme2n10 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 106.0k 106.8k 0.0k 1.0M 1.0M 0.0k nvme2n11 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 74.4k 122.0k 0.0k 744.0k 1.2M 0.0k nvme2n12 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 112.0k 76.4k 0.0k 1.1M 764.0k 0.0k nvme2n13 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 91.6k 85.6k 0.0k 916.0k 856.0k 0.0k nvme2n14 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 100.4k 124.8k 0.0k 1004.0k 1.2M 0.0k nvme2n15 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 98.4k 112.4k 0.0k 984.0k 1.1M 0.0k nvme2n16 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 124.4k 87.2k 0.0k 1.2M 872.0k 0.0k nvme2n17 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 115.6k 94.8k 0.0k 1.1M 948.0k 0.0k nvme2n18 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 96.8k 120.0k 0.0k 968.0k 1.2M 0.0k nvme2n19 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 93.6k 86.4k 0.0k 936.0k 864.0k 0.0k nvme2n2 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 104.0k 150.4k 0.0k 1.0M 1.5M 0.0k nvme2n20 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 102.8k 106.4k 0.0k 1.0M 1.0M 0.0k nvme2n3 2024-05-21T12:37:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 88.8k 117.2k 0.0k 888.0k 1.1M 0.0k nvme2n4 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 104.4k 98.4k 0.0k 1.0M 984.0k 0.0k nvme2n5 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 99.2k 102.0k 0.0k 992.0k 1020.0k 0.0k nvme2n6 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 124.0k 94.0k 0.0k 1.2M 940.0k 0.0k nvme2n7 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 99.2k 116.4k 0.0k 992.0k 1.1M 0.0k nvme2n8 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 113.6k 74.4k 0.0k 1.1M 744.0k 0.0k nvme2n9 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 98.4k 96.4k 0.0k 984.0k 964.0k 0.0k nvme5n1 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 106.0k 98.4k 0.0k 1.0M 984.0k 0.0k nvme5n10 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 118.4k 78.4k 0.0k 1.2M 784.0k 0.0k nvme5n11 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 98.8k 98.4k 0.0k 988.0k 984.0k 0.0k nvme5n12 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 86.4k 125.2k 0.0k 864.0k 1.2M 0.0k nvme5n13 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 96.4k 121.2k 0.0k 964.0k 1.2M 0.0k nvme5n14 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 120.4k 99.6k 0.0k 1.2M 996.0k 0.0k nvme5n15 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 119.6k 104.4k 0.0k 1.2M 1.0M 0.0k nvme5n16 2024-05-21T12:37:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 111.2k 79.6k 0.0k 1.1M 796.0k 0.0k nvme5n17 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 104.4k 112.4k 0.0k 1.0M 1.1M 0.0k nvme5n18 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 110.4k 96.8k 0.0k 1.1M 968.0k 0.0k nvme5n19 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 88.4k 107.2k 0.0k 884.0k 1.0M 0.0k nvme5n2 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 94.0k 103.2k 0.0k 940.0k 1.0M 0.0k nvme5n20 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 118.8k 115.6k 0.0k 1.2M 1.1M 0.0k nvme5n3 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 111.6k 88.4k 0.0k 1.1M 884.0k 0.0k nvme5n4 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 114.8k 90.8k 0.0k 1.1M 908.0k 0.0k nvme5n5 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 106.0k 93.2k 0.0k 1.0M 932.0k 0.0k nvme5n6 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 112.8k 94.0k 0.0k 1.1M 940.0k 0.0k nvme5n7 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 117.2k 118.0k 0.0k 1.1M 1.2M 0.0k nvme5n8 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 104.4k 112.8k 0.0k 1.0M 1.1M 0.0k nvme5n9 2024-05-21T12:37:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 118.4k 90.0k 0.0k 1.2M 900.0k 0.0k nvme8n1 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 93.6k 136.4k 0.0k 936.0k 1.3M 0.0k nvme8n10 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 90.0k 106.0k 0.0k 900.0k 1.0M 0.0k nvme8n11 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 116.8k 99.2k 0.0k 1.1M 992.0k 0.0k nvme8n12 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 117.2k 91.2k 0.0k 1.1M 912.0k 0.0k nvme8n13 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 101.6k 96.8k 0.0k 1016.0k 968.0k 0.0k nvme8n14 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 78.8k 120.4k 0.0k 788.0k 1.2M 0.0k nvme8n15 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 74.8k 120.4k 0.0k 748.0k 1.2M 0.0k nvme8n16 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 117.2k 108.8k 0.0k 1.1M 1.1M 0.0k nvme8n17 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 124.0k 91.6k 0.0k 1.2M 916.0k 0.0k nvme8n18 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 117.2k 104.4k 0.0k 1.1M 1.0M 0.0k nvme8n19 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 102.0k 84.8k 0.0k 1020.0k 848.0k 0.0k nvme8n2 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 93.6k 119.6k 0.0k 936.0k 1.2M 0.0k nvme8n20 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 80.4k 100.4k 0.0k 804.0k 1004.0k 0.0k nvme8n3 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.50 77.2k 116.4k 0.0k 772.0k 1.1M 0.0k nvme8n4 2024-05-21T12:37:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 121.2k 89.6k 0.0k 1.2M 896.0k 0.0k nvme8n5 2024-05-21T12:37:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 98.4k 94.8k 0.0k 984.0k 948.0k 0.0k nvme8n6 2024-05-21T12:37:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 109.2k 110.8k 0.0k 1.1M 1.1M 0.0k nvme8n7 2024-05-21T12:37:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 114.0k 102.4k 0.0k 1.1M 1.0M 0.0k nvme8n8 2024-05-21T12:37:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 10.60 108.4k 114.8k 0.0k 1.1M 1.1M 0.0k nvme8n9 2024-05-21T12:37:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:16 smithi072 ceph-mon[33912]: pgmap v687: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-05-21T12:37:16.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:16 smithi092 ceph-mon[33665]: pgmap v687: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-05-21T12:37:16.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:16 smithi045 ceph-mon[30165]: pgmap v687: 33 pgs: 33 active+clean; 1.0 GiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-05-21T12:37:18.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:18 smithi072 ceph-mon[33912]: pgmap v688: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-05-21T12:37:18.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:18 smithi092 ceph-mon[33665]: pgmap v688: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-05-21T12:37:18.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:18 smithi045 ceph-mon[30165]: pgmap v688: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-05-21T12:37:19.960 INFO:tasks.nvmeof.[nvmeof.thrasher]:done waiting before thrashing 2024-05-21T12:37:19.960 INFO:tasks.nvmeof.[nvmeof.thrasher]:display and verify stats: 2024-05-21T12:37:19.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:37:19.996 INFO:teuthology.orchestra.run.smithi045.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service - Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Active: active (running) since Tue 2024-05-21 12:36:12 UTC; 1min 7s ago 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 75063 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 75064 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Main PID: 75126 (conmon) 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Tasks: 126 (limit: 202854) 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: Memory: 141.7M 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: CPU: 1min 9.558s 2024-05-21T12:37:19.997 INFO:teuthology.orchestra.run.smithi045.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:37:19.998 INFO:teuthology.orchestra.run.smithi045.stdout: ├─libpod-payload-f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 2024-05-21T12:37:19.998 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─75128 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:19.998 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─75130 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:19.998 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─75148 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.a --gateway-address 172.21.15.45:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.45:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.a -k /etc/ceph/keyring 2024-05-21T12:37:19.998 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─75198 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:37:19.998 INFO:teuthology.orchestra.run.smithi045.stdout: │ └─75980 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:19.998 INFO:teuthology.orchestra.run.smithi045.stdout: └─runtime 2024-05-21T12:37:20.001 INFO:teuthology.orchestra.run.smithi045.stdout: └─75126 /usr/bin/conmon --api-version 1 -c f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 -u f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800/userdata -p /run/containers/storage/overlay-containers/f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 2024-05-21T12:37:20.001 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:37:20.001 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 1 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: OPTIMIZED 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: states { 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: grp_id: 1 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: state: OPTIMIZED 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: } 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: ], , client address: IPv4 172.21.15.45:40836 2024-05-21T12:37:20.002 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:36:18 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [2024-05-21 12:36:18.038555] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000279 seconds 2024-05-21T12:37:20.003 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:37:20.039 INFO:teuthology.orchestra.run.smithi072.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service - Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:37:20.039 INFO:teuthology.orchestra.run.smithi072.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:37:20.039 INFO:teuthology.orchestra.run.smithi072.stdout: Active: active (running) since Tue 2024-05-21 12:36:12 UTC; 1min 7s ago 2024-05-21T12:37:20.039 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56908 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:37:20.039 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56909 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:37:20.039 INFO:teuthology.orchestra.run.smithi072.stdout: Main PID: 56971 (conmon) 2024-05-21T12:37:20.039 INFO:teuthology.orchestra.run.smithi072.stdout: Tasks: 126 (limit: 202814) 2024-05-21T12:37:20.039 INFO:teuthology.orchestra.run.smithi072.stdout: Memory: 145.3M 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: CPU: 1min 8.910s 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: ├─libpod-payload-8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56973 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56975 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56993 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.b --gateway-address 172.21.15.72:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.72:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.b -k /etc/ceph/keyring 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─57044 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: │ └─57834 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: └─runtime 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: └─56971 /usr/bin/conmon --api-version 1 -c 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -u 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata -p /run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:37:20.040 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 2 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 2 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:60184 2024-05-21T12:37:20.041 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:36:19 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:36:19.042570] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_2_0 elapsed time: 0.000405 seconds 2024-05-21T12:37:20.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:37:20.075 INFO:teuthology.orchestra.run.smithi092.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service - Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:37:20.075 INFO:teuthology.orchestra.run.smithi092.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:37:20.075 INFO:teuthology.orchestra.run.smithi092.stdout: Active: active (running) since Tue 2024-05-21 12:23:56 UTC; 13min ago 2024-05-21T12:37:20.075 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53502 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:37:20.075 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53503 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:37:20.075 INFO:teuthology.orchestra.run.smithi092.stdout: Main PID: 53679 (conmon) 2024-05-21T12:37:20.075 INFO:teuthology.orchestra.run.smithi092.stdout: Tasks: 126 (limit: 202816) 2024-05-21T12:37:20.075 INFO:teuthology.orchestra.run.smithi092.stdout: Memory: 1.1G 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: CPU: 13min 13.771s 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: ├─libpod-payload-5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53681 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53683 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─53701 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.c --gateway-address 172.21.15.92:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.92:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.c -k /etc/ceph/keyring 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─54233 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: │ └─54267 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: └─runtime 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: └─53679 /usr/bin/conmon --api-version 1 -c 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d -u 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata -p /run/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d 2024-05-21T12:37:20.076 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:37:20.077 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:37:20.077 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:37:20.078 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 1 2024-05-21T12:37:20.078 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:37:20.078 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:37:20.078 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: states { 2024-05-21T12:37:20.078 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: grp_id: 2 2024-05-21T12:37:20.078 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: state: INACCESSIBLE 2024-05-21T12:37:20.078 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: } 2024-05-21T12:37:20.078 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:36:15 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: ], , client address: IPv4 172.21.15.92:59246 2024-05-21T12:37:20.079 DEBUG:teuthology.orchestra.run.smithi137:> ceph orch ls && ceph orch ps --daemon-type nvmeof && ceph health detail && ceph -s && sudo nvme list-subsys /dev/nvme5n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n20 | grep 'live optimized' 2024-05-21T12:37:20.380 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T12:37:20.380 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 1/1 2m ago 18m smithi045=x;count:1 2024-05-21T12:37:20.380 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 2m ago 19m smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:37:20.380 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.mypool ?:4420,5500,8009 1/3 2m ago 13m smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:37:20.380 INFO:teuthology.orchestra.run.smithi137.stdout:osd 8 2m ago - 2024-05-21T12:37:20.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:20 smithi072 ceph-mon[33912]: pgmap v689: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-05-21T12:37:20.681 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:37:20.681 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.a smithi045 *:5500,4420,8009 error 2m ago 12m - - 2024-05-21T12:37:20.681 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.b smithi072 *:5500,4420,8009 error 2m ago 13m - - 2024-05-21T12:37:20.682 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.c smithi092 *:5500,4420,8009 running (13m) 2m ago 13m 160M - 1.2.9 30dd525ace30 5e5a8ffd2ba6 2024-05-21T12:37:20.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:20 smithi092 ceph-mon[33665]: pgmap v689: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-05-21T12:37:20.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:20 smithi045 ceph-mon[30165]: pgmap v689: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-05-21T12:37:21.041 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:37:21.041 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:37:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:37:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:37:21.391 INFO:teuthology.orchestra.run.smithi137.stdout: cluster: 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: health: HEALTH_WARN 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: 2 failed cephadm daemon(s) 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: services: 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: mon: 3 daemons, quorum a,c,b (age 18m) 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: mgr: x(active, since 19m) 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: osd: 8 osds: 8 up (since 15m), 8 in (since 15m) 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: data: 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: pools: 2 pools, 33 pgs 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: objects: 485 objects, 1.0 GiB 2024-05-21T12:37:21.392 INFO:teuthology.orchestra.run.smithi137.stdout: usage: 3.7 GiB used, 712 GiB / 715 GiB avail 2024-05-21T12:37:21.393 INFO:teuthology.orchestra.run.smithi137.stdout: pgs: 33 active+clean 2024-05-21T12:37:21.393 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:37:21.393 INFO:teuthology.orchestra.run.smithi137.stdout: io: 2024-05-21T12:37:21.393 INFO:teuthology.orchestra.run.smithi137.stdout: client: 5.9 MiB/s rd, 5.8 MiB/s wr, 308 op/s rd, 305 op/s wr 2024-05-21T12:37:21.393 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:37:21.423 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.441 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.459 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.477 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.496 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.515 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.533 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.551 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.569 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.588 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.606 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.625 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.653 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:21 smithi092 ceph-mon[33665]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:37:21.653 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:21 smithi092 ceph-mon[33665]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:37:21.653 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:21 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/2638648192' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:37:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:21 smithi072 ceph-mon[33912]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:37:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:21 smithi072 ceph-mon[33912]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:37:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:21 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/2638648192' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:37:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:21 smithi045 ceph-mon[30165]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:37:21.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:21 smithi045 ceph-mon[30165]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:37:21.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:21 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/2638648192' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:37:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.844 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.064 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.174 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.211 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.229 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.284 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.302 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.320 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.340 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.357 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.376 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.412 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.431 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.449 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.486 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.505 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:37:22.505 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.a with skip (0.9856539396040471) > weight (0.3333333333333333) 2024-05-21T12:37:22.505 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.b with skip (0.9251870399173074) > weight (0.3333333333333333) 2024-05-21T12:37:22.505 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.c with skip (0.82484551490352) > weight (0.3333333333333333) 2024-05-21T12:37:22.506 INFO:tasks.nvmeof.[nvmeof.thrasher]:kill nvmeof.a 2024-05-21T12:37:22.506 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Stopping nvmeof.nvmeof.a... 2024-05-21T12:37:22.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:37:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:22 smithi072 ceph-mon[33912]: pgmap v690: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 613 op/s 2024-05-21T12:37:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:22 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/2274255704' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:37:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:22 smithi092 ceph-mon[33665]: pgmap v690: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 613 op/s 2024-05-21T12:37:22.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:22 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/2274255704' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:37:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:22 smithi045 ceph-mon[30165]: pgmap v690: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.8 MiB/s wr, 613 op/s 2024-05-21T12:37:22.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:22 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/2274255704' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:37:22.816 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:22 smithi045 systemd[1]: Stopping Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:37:23.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:37:22] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log.gz 2024-05-21T12:37:24.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:24 smithi045 ceph-mon[30165]: pgmap v691: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.8 MiB/s wr, 672 op/s 2024-05-21T12:37:24.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:24 smithi072 ceph-mon[33912]: pgmap v691: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.8 MiB/s wr, 672 op/s 2024-05-21T12:37:24.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:24 smithi092 ceph-mon[33665]: pgmap v691: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.8 MiB/s wr, 672 op/s 2024-05-21T12:37:24.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 150.0k 113.6k 0.0k 1.5M 1.1M 0.0k nvme2n1 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 175.6k 116.0k 0.0k 1.7M 1.1M 0.0k nvme2n10 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 89.6k 83.6k 0.0k 896.0k 836.0k 0.0k nvme2n11 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 76.0k 77.2k 0.0k 760.0k 772.0k 0.0k nvme2n12 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 128.4k 156.0k 0.0k 1.3M 1.5M 0.0k nvme2n13 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 67.6k 115.2k 0.0k 676.0k 1.1M 0.0k nvme2n14 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 121.6k 117.6k 0.0k 1.2M 1.1M 0.0k nvme2n15 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 90.8k 133.6k 0.0k 908.0k 1.3M 0.0k nvme2n16 2024-05-21T12:37:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 108.8k 105.6k 0.0k 1.1M 1.0M 0.0k nvme2n17 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 152.0k 93.6k 0.0k 1.5M 936.0k 0.0k nvme2n18 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 86.4k 103.2k 0.0k 864.0k 1.0M 0.0k nvme2n19 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 98.8k 137.2k 0.0k 988.0k 1.3M 0.0k nvme2n2 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 147.6k 166.8k 0.0k 1.4M 1.6M 0.0k nvme2n20 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 81.6k 120.0k 0.0k 816.0k 1.2M 0.0k nvme2n3 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 129.6k 166.4k 0.0k 1.3M 1.6M 0.0k nvme2n4 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 110.0k 92.0k 0.0k 1.1M 920.0k 0.0k nvme2n5 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 116.8k 168.8k 0.0k 1.1M 1.6M 0.0k nvme2n6 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 146.4k 158.0k 0.0k 1.4M 1.5M 0.0k nvme2n7 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 132.8k 129.2k 0.0k 1.3M 1.3M 0.0k nvme2n8 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 98.0k 150.4k 0.0k 980.0k 1.5M 0.0k nvme2n9 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 167.6k 170.4k 0.0k 1.6M 1.7M 0.0k nvme5n1 2024-05-21T12:37:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 93.6k 125.2k 0.0k 936.0k 1.2M 0.0k nvme5n10 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 124.4k 136.4k 0.0k 1.2M 1.3M 0.0k nvme5n11 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 68.4k 97.6k 0.0k 684.0k 976.0k 0.0k nvme5n12 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 120.0k 63.2k 0.0k 1.2M 632.0k 0.0k nvme5n13 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 85.2k 78.4k 0.0k 852.0k 784.0k 0.0k nvme5n14 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 92.8k 64.0k 0.0k 928.0k 640.0k 0.0k nvme5n15 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 150.8k 128.8k 0.0k 1.5M 1.3M 0.0k nvme5n16 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 141.6k 138.4k 0.0k 1.4M 1.4M 0.0k nvme5n17 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 98.0k 70.0k 0.0k 980.0k 700.0k 0.0k nvme5n18 2024-05-21T12:37:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 93.2k 101.2k 0.0k 932.0k 1012.0k 0.0k nvme5n19 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 131.2k 136.8k 0.0k 1.3M 1.3M 0.0k nvme5n2 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 79.6k 91.6k 0.0k 796.0k 916.0k 0.0k nvme5n20 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 82.4k 73.6k 0.0k 824.0k 736.0k 0.0k nvme5n3 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 109.2k 74.4k 0.0k 1.1M 744.0k 0.0k nvme5n4 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 108.0k 131.6k 0.0k 1.1M 1.3M 0.0k nvme5n5 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 77.2k 85.2k 0.0k 772.0k 852.0k 0.0k nvme5n6 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 132.4k 138.8k 0.0k 1.3M 1.4M 0.0k nvme5n7 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 99.6k 74.0k 0.0k 996.0k 740.0k 0.0k nvme5n8 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 153.6k 139.2k 0.0k 1.5M 1.4M 0.0k nvme5n9 2024-05-21T12:37:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 89.6k 112.8k 0.0k 896.0k 1.1M 0.0k nvme8n1 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 116.8k 151.2k 0.0k 1.1M 1.5M 0.0k nvme8n10 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 131.2k 100.0k 0.0k 1.3M 1000.0k 0.0k nvme8n11 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 136.0k 111.6k 0.0k 1.3M 1.1M 0.0k nvme8n12 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 84.4k 71.6k 0.0k 844.0k 716.0k 0.0k nvme8n13 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 142.8k 150.8k 0.0k 1.4M 1.5M 0.0k nvme8n14 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 92.8k 82.0k 0.0k 928.0k 820.0k 0.0k nvme8n15 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 139.2k 164.8k 0.0k 1.4M 1.6M 0.0k nvme8n16 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 82.8k 109.6k 0.0k 828.0k 1.1M 0.0k nvme8n17 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 82.4k 80.8k 0.0k 824.0k 808.0k 0.0k nvme8n18 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 88.4k 112.4k 0.0k 884.0k 1.1M 0.0k nvme8n19 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 140.0k 120.0k 0.0k 1.4M 1.2M 0.0k nvme8n2 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 114.0k 153.2k 0.0k 1.1M 1.5M 0.0k nvme8n20 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 110.4k 71.2k 0.0k 1.1M 712.0k 0.0k nvme8n3 2024-05-21T12:37:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 91.2k 79.2k 0.0k 912.0k 792.0k 0.0k nvme8n4 2024-05-21T12:37:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 70.8k 104.0k 0.0k 708.0k 1.0M 0.0k nvme8n5 2024-05-21T12:37:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 144.0k 118.8k 0.0k 1.4M 1.2M 0.0k nvme8n6 2024-05-21T12:37:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 111.6k 130.4k 0.0k 1.1M 1.3M 0.0k nvme8n7 2024-05-21T12:37:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 116.0k 125.6k 0.0k 1.1M 1.2M 0.0k nvme8n8 2024-05-21T12:37:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 72.8k 91.2k 0.0k 728.0k 912.0k 0.0k nvme8n9 2024-05-21T12:37:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:26 smithi072 ceph-mon[33912]: pgmap v692: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-05-21T12:37:26.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:26 smithi092 ceph-mon[33665]: pgmap v692: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-05-21T12:37:26.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:26 smithi045 ceph-mon[30165]: pgmap v692: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-05-21T12:37:28.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:28 smithi072 ceph-mon[33912]: pgmap v693: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 604 op/s 2024-05-21T12:37:28.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:28 smithi092 ceph-mon[33665]: pgmap v693: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 604 op/s 2024-05-21T12:37:28.815 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:28 smithi045 ceph-mon[30165]: pgmap v693: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 604 op/s 2024-05-21T12:37:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:30 smithi072 ceph-mon[33912]: pgmap v694: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.5 MiB/s wr, 551 op/s 2024-05-21T12:37:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:30 smithi092 ceph-mon[33665]: pgmap v694: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.5 MiB/s wr, 551 op/s 2024-05-21T12:37:30.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:30 smithi045 ceph-mon[30165]: pgmap v694: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.5 MiB/s wr, 551 op/s 2024-05-21T12:37:32.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:32 smithi072 ceph-mon[33912]: pgmap v695: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 561 op/s 2024-05-21T12:37:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:32 smithi092 ceph-mon[33665]: pgmap v695: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 561 op/s 2024-05-21T12:37:32.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:32 smithi045 ceph-mon[30165]: pgmap v695: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 561 op/s 2024-05-21T12:37:33.279 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:32 smithi045 bash[76434]: time="2024-05-21T12:37:32Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a in 10 seconds, resorting to SIGKILL" 2024-05-21T12:37:33.279 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:32 smithi045 podman[76434]: 2024-05-21 12:37:32.879310919 +0000 UTC m=+10.164228040 container died f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, distribution-scope=public, version=1.2.9, io.spdk.git.branch=undefined, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.git.branch=tags/1.2.9, io.ceph.version=1.2.9, io.ceph.component=ceph-nvmeof, io.ceph.maintainer=Ceph Developers , vcs-type=git, build-date=2024-05-19 17:57:19 UTC, name=ceph-nvmeof, io.spdk.build.ceph-release=19.0.0, release=, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.build.target-arch=, io.spdk.version=24.01, vendor=Red Hat, Inc., io.ceph.nvmeof=None, io.buildah.version=1.29.0, summary=Ceph NVMe over Fabrics Gateway, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, org.centos.version=9.0-21.el9, io.spdk.build.makeflags=, architecture=x86_64, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.repo=https://github.com/ceph/spdk.git, url=https://github.com/ceph/ceph-nvmeof, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, com.redhat.component=ubi9-container, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers ) 2024-05-21T12:37:33.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:33 smithi045 podman[76434]: 2024-05-21 12:37:33.496074996 +0000 UTC m=+10.780992108 container cleanup f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.spdk.build.makeflags=, io.spdk.build.target-arch=, io.spdk.git.branch=undefined, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.buildah.version=1.29.0, io.ceph.version=1.2.9, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.build.pkgdep-args=--rbd, summary=Ceph NVMe over Fabrics Gateway, build-date=2024-05-19 17:57:19 UTC, version=1.2.9, architecture=x86_64, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, name=ceph-nvmeof, org.centos.version=9.0-21.el9, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, io.ceph.component=ceph-nvmeof, distribution-scope=public, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, release=, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.maintainer=Ceph Developers , io.spdk.build.ceph-release=19.0.0, vcs-type=git, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.git.branch=tags/1.2.9, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.nvmeof=None, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi9-container, maintainer=Ceph Developers , io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.version=24.01) 2024-05-21T12:37:33.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:33 smithi045 bash[76434]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a 2024-05-21T12:37:33.992 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service' 2024-05-21T12:37:34.016 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:33 smithi045 podman[76477]: 2024-05-21 12:37:33.747621632 +0000 UTC m=+0.866904792 container remove f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vendor=Red Hat, Inc., build-date=2024-05-19 17:57:19 UTC, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, url=https://github.com/ceph/ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.component=ceph-nvmeof, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.version=24.01, release=, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.target-arch=, io.buildah.version=1.29.0, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vcs-type=git, io.ceph.version=1.2.9, name=ceph-nvmeof, distribution-scope=public, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.repo=https://github.com/ceph/spdk.git, com.redhat.component=ubi9-container, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.maintainer=Ceph Developers , io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.git.branch=tags/1.2.9, io.spdk.build.ceph-release=19.0.0, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.nvmeof=None, maintainer=Ceph Developers , io.spdk.build.pkgdep-args=--rbd, version=1.2.9, architecture=x86_64, org.centos.version=9.0-21.el9, summary=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.git.branch=undefined) 2024-05-21T12:37:34.016 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:33 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:37:34.016 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:33 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Failed with result 'exit-code'. 2024-05-21T12:37:34.016 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:33 smithi045 systemd[1]: Stopped Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:37:34.016 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:37:33 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Consumed 1min 23.403s CPU time. 2024-05-21T12:37:34.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:37:34.031 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Stopped nvmeof.nvmeof.a 2024-05-21T12:37:34.031 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.b with skip (0.7239356507653069) > weight (0.3333333333333333) 2024-05-21T12:37:34.031 INFO:tasks.nvmeof.[nvmeof.thrasher]:kill nvmeof.c 2024-05-21T12:37:34.031 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.c:Stopping nvmeof.nvmeof.c... 2024-05-21T12:37:34.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:37:34.354 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:37:34 smithi092 systemd[1]: Stopping Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:37:34.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:34 smithi072 ceph-mon[33912]: pgmap v696: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.3 MiB/s wr, 623 op/s 2024-05-21T12:37:34.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:34 smithi045 ceph-mon[30165]: pgmap v696: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.3 MiB/s wr, 623 op/s 2024-05-21T12:37:34.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:34 smithi092 ceph-mon[33665]: pgmap v696: 33 pgs: 33 active+clean; 1.0 GiB data, 3.7 GiB used, 711 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.3 MiB/s wr, 623 op/s 2024-05-21T12:37:34.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:37:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:37:34] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log.gz 2024-05-21T12:37:34.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 111.2k 120.8k 0.0k 1.1M 1.2M 0.0k nvme2n1 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 114.0k 109.6k 0.0k 1.1M 1.1M 0.0k nvme2n10 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 73.2k 104.4k 0.0k 732.0k 1.0M 0.0k nvme2n11 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 106.4k 89.2k 0.0k 1.0M 892.0k 0.0k nvme2n12 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 89.2k 128.4k 0.0k 892.0k 1.3M 0.0k nvme2n13 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 113.6k 130.8k 0.0k 1.1M 1.3M 0.0k nvme2n14 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 112.8k 112.8k 0.0k 1.1M 1.1M 0.0k nvme2n15 2024-05-21T12:37:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 105.6k 133.6k 0.0k 1.0M 1.3M 0.0k nvme2n16 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 120.4k 101.6k 0.0k 1.2M 1016.0k 0.0k nvme2n17 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 70.8k 165.2k 0.0k 708.0k 1.6M 0.0k nvme2n18 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 115.2k 138.8k 0.0k 1.1M 1.4M 0.0k nvme2n19 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 141.2k 108.8k 0.0k 1.4M 1.1M 0.0k nvme2n2 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 118.0k 103.2k 0.0k 1.2M 1.0M 0.0k nvme2n20 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 97.2k 154.8k 0.0k 972.0k 1.5M 0.0k nvme2n3 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 134.0k 127.2k 0.0k 1.3M 1.2M 0.0k nvme2n4 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 121.6k 130.0k 0.0k 1.2M 1.3M 0.0k nvme2n5 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 104.0k 135.6k 0.0k 1.0M 1.3M 0.0k nvme2n6 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 93.2k 147.6k 0.0k 932.0k 1.4M 0.0k nvme2n7 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 90.4k 113.6k 0.0k 904.0k 1.1M 0.0k nvme2n8 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 138.8k 97.6k 0.0k 1.4M 976.0k 0.0k nvme2n9 2024-05-21T12:37:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 94.8k 120.8k 0.0k 948.0k 1.2M 0.0k nvme5n1 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 125.2k 117.2k 0.0k 1.2M 1.1M 0.0k nvme5n10 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 135.6k 98.8k 0.0k 1.3M 988.0k 0.0k nvme5n11 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 112.0k 72.8k 0.0k 1.1M 728.0k 0.0k nvme5n12 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 172.8k 75.2k 0.0k 1.7M 752.0k 0.0k nvme5n13 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 85.6k 76.8k 0.0k 856.0k 768.0k 0.0k nvme5n14 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 109.2k 86.0k 0.0k 1.1M 860.0k 0.0k nvme5n15 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 127.2k 102.8k 0.0k 1.2M 1.0M 0.0k nvme5n16 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 110.8k 146.0k 0.0k 1.1M 1.4M 0.0k nvme5n17 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 137.6k 126.4k 0.0k 1.3M 1.2M 0.0k nvme5n18 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 134.0k 84.8k 0.0k 1.3M 848.0k 0.0k nvme5n19 2024-05-21T12:37:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 122.8k 106.0k 0.0k 1.2M 1.0M 0.0k nvme5n2 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 132.8k 133.6k 0.0k 1.3M 1.3M 0.0k nvme5n20 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 104.8k 87.6k 0.0k 1.0M 876.0k 0.0k nvme5n3 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 86.8k 88.4k 0.0k 868.0k 884.0k 0.0k nvme5n4 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 138.0k 113.6k 0.0k 1.3M 1.1M 0.0k nvme5n5 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 139.2k 108.0k 0.0k 1.4M 1.1M 0.0k nvme5n6 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 158.4k 65.2k 0.0k 1.5M 652.0k 0.0k nvme5n7 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 125.6k 129.6k 0.0k 1.2M 1.3M 0.0k nvme5n8 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 97.6k 103.6k 0.0k 976.0k 1.0M 0.0k nvme5n9 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 112.4k 110.0k 0.0k 1.1M 1.1M 0.0k nvme8n1 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 101.6k 104.4k 0.0k 1016.0k 1.0M 0.0k nvme8n10 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 134.8k 112.0k 0.0k 1.3M 1.1M 0.0k nvme8n11 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 102.0k 106.0k 0.0k 1020.0k 1.0M 0.0k nvme8n12 2024-05-21T12:37:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 89.6k 118.0k 0.0k 896.0k 1.2M 0.0k nvme8n13 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 135.2k 112.0k 0.0k 1.3M 1.1M 0.0k nvme8n14 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 70.4k 87.2k 0.0k 704.0k 872.0k 0.0k nvme8n15 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 132.0k 83.6k 0.0k 1.3M 836.0k 0.0k nvme8n16 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 102.8k 128.8k 0.0k 1.0M 1.3M 0.0k nvme8n17 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 142.4k 106.8k 0.0k 1.4M 1.0M 0.0k nvme8n18 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 111.6k 117.6k 0.0k 1.1M 1.1M 0.0k nvme8n19 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 118.4k 117.2k 0.0k 1.2M 1.1M 0.0k nvme8n2 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 89.2k 124.8k 0.0k 892.0k 1.2M 0.0k nvme8n20 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 88.0k 104.0k 0.0k 880.0k 1.0M 0.0k nvme8n3 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 134.0k 96.8k 0.0k 1.3M 968.0k 0.0k nvme8n4 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 104.0k 156.8k 0.0k 1.0M 1.5M 0.0k nvme8n5 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 98.4k 141.2k 0.0k 984.0k 1.4M 0.0k nvme8n6 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 104.0k 138.8k 0.0k 1.0M 1.4M 0.0k nvme8n7 2024-05-21T12:37:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 142.0k 97.2k 0.0k 1.4M 972.0k 0.0k nvme8n8 2024-05-21T12:37:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 121.6k 117.2k 0.0k 1.2M 1.1M 0.0k nvme8n9 2024-05-21T12:37:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:36.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:36 smithi072 ceph-mon[33912]: pgmap v697: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 537 op/s 2024-05-21T12:37:36.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:36 smithi092 ceph-mon[33665]: pgmap v697: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 537 op/s 2024-05-21T12:37:36.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:36 smithi045 ceph-mon[30165]: pgmap v697: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 537 op/s 2024-05-21T12:37:38.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:38 smithi072 ceph-mon[33912]: pgmap v698: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 469 op/s 2024-05-21T12:37:38.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:38 smithi092 ceph-mon[33665]: pgmap v698: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 469 op/s 2024-05-21T12:37:38.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:38 smithi045 ceph-mon[30165]: pgmap v698: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 469 op/s 2024-05-21T12:37:40.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:40 smithi072 ceph-mon[33912]: pgmap v699: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-05-21T12:37:40.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:40 smithi092 ceph-mon[33665]: pgmap v699: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-05-21T12:37:40.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:40 smithi045 ceph-mon[30165]: pgmap v699: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 369 op/s 2024-05-21T12:37:42.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:42 smithi072 ceph-mon[33912]: pgmap v700: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 306 op/s 2024-05-21T12:37:42.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:42 smithi092 ceph-mon[33665]: pgmap v700: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 306 op/s 2024-05-21T12:37:42.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:42 smithi045 ceph-mon[30165]: pgmap v700: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 306 op/s 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:37:43] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:43.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:37:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:37:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:48108 2024-05-21T12:37:43.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:43 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:37:43.060483] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000237 seconds 2024-05-21T12:37:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:44 smithi045 ceph-mon[30165]: osdmap e101: 8 total, 8 up, 8 in 2024-05-21T12:37:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:44 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:44 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:44 smithi045 ceph-mon[30165]: pgmap v702: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-21T12:37:44.385 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:44 smithi092 ceph-mon[33665]: osdmap e101: 8 total, 8 up, 8 in 2024-05-21T12:37:44.385 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:44 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:44.385 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:44 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:44.385 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:44 smithi092 ceph-mon[33665]: pgmap v702: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-21T12:37:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-mon[33912]: osdmap e101: 8 total, 8 up, 8 in 2024-05-21T12:37:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-mon[33912]: pgmap v702: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:37:44] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:44.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:37:44.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:44.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:37:44.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:44.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:44.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:48130 2024-05-21T12:37:44.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:44 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:37:44.068535] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000208 seconds 2024-05-21T12:37:44.708 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:37:44 smithi092 bash[57676]: time="2024-05-21T12:37:44Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c in 10 seconds, resorting to SIGKILL" 2024-05-21T12:37:44.708 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:37:44 smithi092 podman[57676]: 2024-05-21 12:37:44.403094466 +0000 UTC m=+10.186405340 container died 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.ceph.component=ceph-nvmeof, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, url=https://github.com/ceph/ceph-nvmeof, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, architecture=x86_64, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, org.centos.version=9.0-21.el9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, vcs-type=git, name=ceph-nvmeof, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vendor=Red Hat, Inc., com.redhat.component=ubi9-container, release=, io.openshift.wants=ceph,rbd, distribution-scope=public, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.git.repo=https://github.com/ceph/spdk.git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.ceph-release=19.0.0, io.ceph.maintainer=Ceph Developers , maintainer=Ceph Developers , version=1.2.9, build-date=2024-05-19 17:57:19 UTC, io.ceph.nvmeof=None, io.buildah.version=1.29.0, io.spdk.git.branch=undefined, io.spdk.build.pkgdep-args=--rbd, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.version=1.2.9, io.spdk.build.target-arch=, summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, io.spdk.version=24.01, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.ceph.git.branch=tags/1.2.9) 2024-05-21T12:37:44.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n1 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n10 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n11 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n12 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n13 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n14 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n15 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n16 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n17 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n18 2024-05-21T12:37:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n19 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n2 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n20 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n3 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n4 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n5 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n6 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n7 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n8 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n9 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n1 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n10 2024-05-21T12:37:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n11 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n12 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n13 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n14 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n15 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n16 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n17 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme5n18 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n19 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n2 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n20 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n3 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n4 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n5 2024-05-21T12:37:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n6 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n7 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n8 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n1 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n10 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n11 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n12 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n13 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n14 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n15 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n16 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme8n17 2024-05-21T12:37:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n18 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n19 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n2 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n20 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n3 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n4 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n5 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n6 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n7 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n9 2024-05-21T12:37:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:45.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:37:44 smithi092 podman[57676]: 2024-05-21 12:37:44.857670501 +0000 UTC m=+10.640981385 container cleanup 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.spdk.build.makeflags=, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., release=, io.ceph.maintainer=Ceph Developers , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.git.branch=tags/1.2.9, architecture=x86_64, io.ceph.component=ceph-nvmeof, vendor=Red Hat, Inc., io.ceph.version=1.2.9, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., version=1.2.9, build-date=2024-05-19 17:57:19 UTC, com.redhat.component=ubi9-container, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, summary=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vcs-type=git, io.openshift.wants=ceph,rbd, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.build.target-arch=, io.spdk.git.branch=undefined, io.spdk.version=24.01, name=ceph-nvmeof, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.nvmeof=None, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.ceph-release=19.0.0, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.buildah.version=1.29.0, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, distribution-scope=public, org.centos.version=9.0-21.el9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.pkgdep-args=--rbd, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.url=https://github.com/ceph/ceph-nvmeof, maintainer=Ceph Developers , url=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:37:45.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:37:44 smithi092 bash[57676]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c 2024-05-21T12:37:45.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service' 2024-05-21T12:37:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:45 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:45 smithi045 ceph-mon[30165]: osdmap e102: 8 total, 8 up, 8 in 2024-05-21T12:37:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:45 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:45 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:45.324 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:45 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:45.324 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:45 smithi092 ceph-mon[33665]: osdmap e102: 8 total, 8 up, 8 in 2024-05-21T12:37:45.324 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:45 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:45.324 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:45 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:45.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:37:45.325 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.c:Stopped nvmeof.nvmeof.c 2024-05-21T12:37:45.326 INFO:tasks.nvmeof.[nvmeof.thrasher]:waiting for 123 secs before reviving 2024-05-21T12:37:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:45 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:45 smithi072 ceph-mon[33912]: osdmap e102: 8 total, 8 up, 8 in 2024-05-21T12:37:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:45 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:45 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:46.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:46 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:46.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:46 smithi072 ceph-mon[33912]: osdmap e103: 8 total, 8 up, 8 in 2024-05-21T12:37:46.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:46 smithi072 ceph-mon[33912]: pgmap v705: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-05-21T12:37:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:46 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:46 smithi092 ceph-mon[33665]: osdmap e103: 8 total, 8 up, 8 in 2024-05-21T12:37:46.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:46 smithi092 ceph-mon[33665]: pgmap v705: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-05-21T12:37:46.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:46 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:46.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:46 smithi045 ceph-mon[30165]: osdmap e103: 8 total, 8 up, 8 in 2024-05-21T12:37:46.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:46 smithi045 ceph-mon[30165]: pgmap v705: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-05-21T12:37:48.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:48 smithi072 ceph-mon[33912]: pgmap v706: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-05-21T12:37:48.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:48 smithi092 ceph-mon[33665]: pgmap v706: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-05-21T12:37:48.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:48 smithi045 ceph-mon[30165]: pgmap v706: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-05-21T12:37:50.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:50 smithi072 ceph-mon[33912]: pgmap v707: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-05-21T12:37:50.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:50 smithi092 ceph-mon[33665]: pgmap v707: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-05-21T12:37:50.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:50 smithi045 ceph-mon[30165]: pgmap v707: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-05-21T12:37:52.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:52 smithi072 ceph-mon[33912]: pgmap v708: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 9.4 KiB/s wr, 5 op/s 2024-05-21T12:37:52.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:52 smithi092 ceph-mon[33665]: pgmap v708: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 9.4 KiB/s wr, 5 op/s 2024-05-21T12:37:52.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:52 smithi045 ceph-mon[30165]: pgmap v708: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 9.4 KiB/s wr, 5 op/s 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:37:53] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:37:53.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:53.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:53.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:37:53.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:53.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:37:53.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:53.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:53.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:49850 2024-05-21T12:37:53.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:37:53.061486] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000479 seconds 2024-05-21T12:37:54.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:54 smithi045 ceph-mon[30165]: osdmap e104: 8 total, 8 up, 8 in 2024-05-21T12:37:54.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:54 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:54.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:54 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:54.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:54 smithi045 ceph-mon[30165]: pgmap v710: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 7.2 KiB/s wr, 5 op/s 2024-05-21T12:37:54.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-mon[33912]: osdmap e104: 8 total, 8 up, 8 in 2024-05-21T12:37:54.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:54.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-mon[33912]: pgmap v710: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 7.2 KiB/s wr, 5 op/s 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:37:54] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:37:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:37:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:37:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:37:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:37:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:37:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:49880 2024-05-21T12:37:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:37:54.065336] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000217 seconds 2024-05-21T12:37:54.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:54 smithi092 ceph-mon[33665]: osdmap e104: 8 total, 8 up, 8 in 2024-05-21T12:37:54.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:54 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:54.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:54 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:54.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:54 smithi092 ceph-mon[33665]: pgmap v710: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 7.2 KiB/s wr, 5 op/s 2024-05-21T12:37:54.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:37:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme2n1 2024-05-21T12:37:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n10 2024-05-21T12:37:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n11 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n12 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n13 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n14 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n15 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n16 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n17 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n18 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.6k 0.0k 0.0k 56.0k 0.0k 0.0k nvme2n19 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme2n2 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme2n20 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n3 2024-05-21T12:37:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n4 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n5 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n6 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n7 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n8 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n9 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.2k 0.0k 0.0k 52.0k 0.0k 0.0k nvme5n1 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n10 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n11 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n12 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n13 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n14 2024-05-21T12:37:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n15 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n16 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n17 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n18 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme5n19 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme5n2 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.4k 0.0k 0.0k 44.0k 0.0k nvme5n20 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n3 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n4 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n5 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n6 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n7 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n8 2024-05-21T12:37:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme8n1 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n10 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n11 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n12 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n13 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n14 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n15 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n16 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n17 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme8n18 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n19 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n2 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.2k 0.0k 0.0k 12.0k 0.0k 0.0k nvme8n20 2024-05-21T12:37:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n3 2024-05-21T12:37:54.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n4 2024-05-21T12:37:54.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n5 2024-05-21T12:37:54.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n6 2024-05-21T12:37:54.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n7 2024-05-21T12:37:54.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:37:54.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n9 2024-05-21T12:37:54.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:37:55.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:55 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:55.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:55 smithi045 ceph-mon[30165]: osdmap e105: 8 total, 8 up, 8 in 2024-05-21T12:37:55.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:55 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:55.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:55 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:55.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:55 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:55.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:55 smithi072 ceph-mon[33912]: osdmap e105: 8 total, 8 up, 8 in 2024-05-21T12:37:55.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:55 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:55.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:55 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:55.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:55 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:55.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:55 smithi092 ceph-mon[33665]: osdmap e105: 8 total, 8 up, 8 in 2024-05-21T12:37:55.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:55 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:55.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:55 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:56 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:56 smithi072 ceph-mon[33912]: osdmap e106: 8 total, 8 up, 8 in 2024-05-21T12:37:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:56 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:56 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:56 smithi072 ceph-mon[33912]: pgmap v713: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 3 op/s 2024-05-21T12:37:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:56 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:37:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:56 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:56 smithi092 ceph-mon[33665]: osdmap e106: 8 total, 8 up, 8 in 2024-05-21T12:37:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:56 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:56 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:56 smithi092 ceph-mon[33665]: pgmap v713: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 3 op/s 2024-05-21T12:37:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:56 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:37:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:56 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:56 smithi045 ceph-mon[30165]: osdmap e106: 8 total, 8 up, 8 in 2024-05-21T12:37:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:56 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:56 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:56 smithi045 ceph-mon[30165]: pgmap v713: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 3 op/s 2024-05-21T12:37:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:56 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:37:57.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:57.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: osdmap e107: 8 total, 8 up, 8 in 2024-05-21T12:37:57.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:37:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:37:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: pgmap v715: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 69 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-21T12:37:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: pgmap v716: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-05-21T12:37:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: osdmap e107: 8 total, 8 up, 8 in 2024-05-21T12:37:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:57.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:37:57.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:37:57.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: pgmap v715: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 69 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-21T12:37:57.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: pgmap v716: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-05-21T12:37:57.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: osdmap e107: 8 total, 8 up, 8 in 2024-05-21T12:37:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:57.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:57.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:57.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:37:57.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:37:57.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: pgmap v715: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 69 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-21T12:37:57.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: pgmap v716: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-05-21T12:37:57.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:37:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:58 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:58 smithi072 ceph-mon[33912]: osdmap e108: 8 total, 8 up, 8 in 2024-05-21T12:37:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:58 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:58 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:58 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:58 smithi092 ceph-mon[33665]: osdmap e108: 8 total, 8 up, 8 in 2024-05-21T12:37:58.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:58 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:58.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:58 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:58 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:37:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:58 smithi045 ceph-mon[30165]: osdmap e108: 8 total, 8 up, 8 in 2024-05-21T12:37:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:58 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:58 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:37:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:59 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:59 smithi072 ceph-mon[33912]: osdmap e109: 8 total, 8 up, 8 in 2024-05-21T12:37:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:59 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:59 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:37:59 smithi072 ceph-mon[33912]: pgmap v719: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 99 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-05-21T12:37:59.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:59 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:59 smithi092 ceph-mon[33665]: osdmap e109: 8 total, 8 up, 8 in 2024-05-21T12:37:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:59 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:59 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:37:59 smithi092 ceph-mon[33665]: pgmap v719: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 99 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-05-21T12:37:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:59 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:37:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:59 smithi045 ceph-mon[30165]: osdmap e109: 8 total, 8 up, 8 in 2024-05-21T12:37:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:59 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:59 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:37:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:37:59 smithi045 ceph-mon[30165]: pgmap v719: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 99 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-05-21T12:38:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:00 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:00 smithi072 ceph-mon[33912]: osdmap e110: 8 total, 8 up, 8 in 2024-05-21T12:38:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:00 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:00 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:00 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:00.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:00 smithi092 ceph-mon[33665]: osdmap e110: 8 total, 8 up, 8 in 2024-05-21T12:38:00.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:00 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:00.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:00 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:00 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:00 smithi045 ceph-mon[30165]: osdmap e110: 8 total, 8 up, 8 in 2024-05-21T12:38:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:00 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:00 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:01 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:01 smithi072 ceph-mon[33912]: osdmap e111: 8 total, 8 up, 8 in 2024-05-21T12:38:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:01 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:01 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:01 smithi072 ceph-mon[33912]: pgmap v722: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 48 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-05-21T12:38:01.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:01 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:01.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:01 smithi092 ceph-mon[33665]: osdmap e111: 8 total, 8 up, 8 in 2024-05-21T12:38:01.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:01 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:01.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:01 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:01.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:01 smithi092 ceph-mon[33665]: pgmap v722: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 48 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-05-21T12:38:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:01 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:01 smithi045 ceph-mon[30165]: osdmap e111: 8 total, 8 up, 8 in 2024-05-21T12:38:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:01 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:01 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:01 smithi045 ceph-mon[30165]: pgmap v722: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 48 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-05-21T12:38:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:02 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:02 smithi072 ceph-mon[33912]: osdmap e112: 8 total, 8 up, 8 in 2024-05-21T12:38:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:02 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:02 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:02.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:02 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:02 smithi092 ceph-mon[33665]: osdmap e112: 8 total, 8 up, 8 in 2024-05-21T12:38:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:02 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:02 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:02 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:02 smithi045 ceph-mon[30165]: osdmap e112: 8 total, 8 up, 8 in 2024-05-21T12:38:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:02 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:02 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:03 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:03 smithi072 ceph-mon[33912]: osdmap e113: 8 total, 8 up, 8 in 2024-05-21T12:38:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:03 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:03 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:03.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:03 smithi072 ceph-mon[33912]: pgmap v725: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-05-21T12:38:03.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:03 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:03.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:03 smithi092 ceph-mon[33665]: osdmap e113: 8 total, 8 up, 8 in 2024-05-21T12:38:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:03 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:03 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:03 smithi092 ceph-mon[33665]: pgmap v725: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-05-21T12:38:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:03 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:03 smithi045 ceph-mon[30165]: osdmap e113: 8 total, 8 up, 8 in 2024-05-21T12:38:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:03 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:03 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:03 smithi045 ceph-mon[30165]: pgmap v725: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-05-21T12:38:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:04 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:04 smithi072 ceph-mon[33912]: osdmap e114: 8 total, 8 up, 8 in 2024-05-21T12:38:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:04 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:04 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:04 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:04 smithi092 ceph-mon[33665]: osdmap e114: 8 total, 8 up, 8 in 2024-05-21T12:38:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:04 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:04 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:04 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:04 smithi045 ceph-mon[30165]: osdmap e114: 8 total, 8 up, 8 in 2024-05-21T12:38:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:04 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:04 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:04.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:38:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n1 2024-05-21T12:38:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme2n10 2024-05-21T12:38:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n11 2024-05-21T12:38:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n12 2024-05-21T12:38:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n13 2024-05-21T12:38:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n14 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n15 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n16 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n17 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n18 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n19 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n2 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n20 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme2n3 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme2n4 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.4k 0.0k 0.0k 64.0k 0.0k nvme2n5 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme2n6 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme2n7 2024-05-21T12:38:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.4k 0.0k 0.0k 64.0k 0.0k 0.0k nvme2n8 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme2n9 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n1 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme5n10 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n11 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n12 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n13 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n14 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n15 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n16 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n17 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n18 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n19 2024-05-21T12:38:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n2 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n20 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme5n3 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme5n4 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme5n5 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme5n6 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme5n7 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme5n8 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme5n9 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n1 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme8n10 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n11 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n12 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n13 2024-05-21T12:38:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n14 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n15 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n16 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n17 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n18 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n19 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.4k 0.0k 0.0k 64.0k 0.0k 0.0k nvme8n2 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n20 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme8n3 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.6k 0.0k 0.0k 56.0k 0.0k 0.0k nvme8n4 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme8n5 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.0k 0.0k 0.0k 60.0k 0.0k 0.0k nvme8n6 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.4k 0.0k 0.0k 64.0k 0.0k 0.0k nvme8n7 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme8n8 2024-05-21T12:38:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme8n9 2024-05-21T12:38:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:05.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:05 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:05.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:05 smithi072 ceph-mon[33912]: osdmap e115: 8 total, 8 up, 8 in 2024-05-21T12:38:05.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:05 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:05.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:05 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:05.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:05 smithi072 ceph-mon[33912]: pgmap v728: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-05-21T12:38:05.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:05 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:05 smithi092 ceph-mon[33665]: osdmap e115: 8 total, 8 up, 8 in 2024-05-21T12:38:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:05 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:05 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:05 smithi092 ceph-mon[33665]: pgmap v728: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-05-21T12:38:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:05 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:05 smithi045 ceph-mon[30165]: osdmap e115: 8 total, 8 up, 8 in 2024-05-21T12:38:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:05 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:05.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:05 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:05.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:05 smithi045 ceph-mon[30165]: pgmap v728: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 74 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-05-21T12:38:06.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:06 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:06.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:06 smithi072 ceph-mon[33912]: osdmap e116: 8 total, 8 up, 8 in 2024-05-21T12:38:06.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:06 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:06.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:06 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:06 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:06.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:06 smithi092 ceph-mon[33665]: osdmap e116: 8 total, 8 up, 8 in 2024-05-21T12:38:06.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:06 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:06.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:06 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:06 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:06 smithi045 ceph-mon[30165]: osdmap e116: 8 total, 8 up, 8 in 2024-05-21T12:38:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:06 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:06 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:07.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:07 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:07.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:07 smithi072 ceph-mon[33912]: osdmap e117: 8 total, 8 up, 8 in 2024-05-21T12:38:07.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:07 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:07.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:07 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:07.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:07 smithi072 ceph-mon[33912]: pgmap v731: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 51 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-05-21T12:38:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:07 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:07 smithi092 ceph-mon[33665]: osdmap e117: 8 total, 8 up, 8 in 2024-05-21T12:38:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:07 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:07 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:07 smithi092 ceph-mon[33665]: pgmap v731: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 51 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-05-21T12:38:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:07 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:07 smithi045 ceph-mon[30165]: osdmap e117: 8 total, 8 up, 8 in 2024-05-21T12:38:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:07 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:07 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:07 smithi045 ceph-mon[30165]: pgmap v731: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 51 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-05-21T12:38:08.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:08 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:08.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:08 smithi072 ceph-mon[33912]: osdmap e118: 8 total, 8 up, 8 in 2024-05-21T12:38:08.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:08 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:08.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:08 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:08 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:08 smithi092 ceph-mon[33665]: osdmap e118: 8 total, 8 up, 8 in 2024-05-21T12:38:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:08 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:08 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:08 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:08 smithi045 ceph-mon[30165]: osdmap e118: 8 total, 8 up, 8 in 2024-05-21T12:38:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:08 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:08 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:09.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:09 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:09.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:09 smithi072 ceph-mon[33912]: osdmap e119: 8 total, 8 up, 8 in 2024-05-21T12:38:09.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:09 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:09.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:09 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:09.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:09 smithi072 ceph-mon[33912]: pgmap v734: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-05-21T12:38:09.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:09 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:09.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:09 smithi092 ceph-mon[33665]: osdmap e119: 8 total, 8 up, 8 in 2024-05-21T12:38:09.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:09 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:09.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:09 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:09.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:09 smithi092 ceph-mon[33665]: pgmap v734: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-05-21T12:38:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:09 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:09 smithi045 ceph-mon[30165]: osdmap e119: 8 total, 8 up, 8 in 2024-05-21T12:38:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:09 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:09 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:09 smithi045 ceph-mon[30165]: pgmap v734: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 83 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-05-21T12:38:10.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:10 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:10.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:10 smithi072 ceph-mon[33912]: osdmap e120: 8 total, 8 up, 8 in 2024-05-21T12:38:10.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:10 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:10.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:10 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:10.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:10 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:10.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:10 smithi092 ceph-mon[33665]: osdmap e120: 8 total, 8 up, 8 in 2024-05-21T12:38:10.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:10 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:10.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:10 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:10 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:10 smithi045 ceph-mon[30165]: osdmap e120: 8 total, 8 up, 8 in 2024-05-21T12:38:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:10 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:10 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:11.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:11 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:11.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:11 smithi072 ceph-mon[33912]: osdmap e121: 8 total, 8 up, 8 in 2024-05-21T12:38:11.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:11 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:11.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:11 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:11.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:11 smithi072 ceph-mon[33912]: pgmap v737: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 109 KiB/s rd, 70 KiB/s wr, 14 op/s 2024-05-21T12:38:11.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:11 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:11.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:11 smithi092 ceph-mon[33665]: osdmap e121: 8 total, 8 up, 8 in 2024-05-21T12:38:11.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:11 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:11.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:11 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:11.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:11 smithi092 ceph-mon[33665]: pgmap v737: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 109 KiB/s rd, 70 KiB/s wr, 14 op/s 2024-05-21T12:38:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:11 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:11 smithi045 ceph-mon[30165]: osdmap e121: 8 total, 8 up, 8 in 2024-05-21T12:38:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:11 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:11 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:11 smithi045 ceph-mon[30165]: pgmap v737: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 109 KiB/s rd, 70 KiB/s wr, 14 op/s 2024-05-21T12:38:12.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:12 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:12.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:12 smithi072 ceph-mon[33912]: osdmap e122: 8 total, 8 up, 8 in 2024-05-21T12:38:12.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:12 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:12.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:12 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:12.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:12 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:12.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:12 smithi092 ceph-mon[33665]: osdmap e122: 8 total, 8 up, 8 in 2024-05-21T12:38:12.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:12 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:12.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:12 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:12 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:12 smithi045 ceph-mon[30165]: osdmap e122: 8 total, 8 up, 8 in 2024-05-21T12:38:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:12 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:12 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:13 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:13 smithi092 ceph-mon[33665]: osdmap e123: 8 total, 8 up, 8 in 2024-05-21T12:38:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:13 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:13 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:13 smithi092 ceph-mon[33665]: pgmap v740: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 117 KiB/s rd, 62 KiB/s wr, 30 op/s 2024-05-21T12:38:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:13 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:13 smithi045 ceph-mon[30165]: osdmap e123: 8 total, 8 up, 8 in 2024-05-21T12:38:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:13 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:13 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:13 smithi045 ceph-mon[30165]: pgmap v740: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 117 KiB/s rd, 62 KiB/s wr, 30 op/s 2024-05-21T12:38:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:13 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:13 smithi072 ceph-mon[33912]: osdmap e123: 8 total, 8 up, 8 in 2024-05-21T12:38:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:13 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:13 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:13 smithi072 ceph-mon[33912]: pgmap v740: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 117 KiB/s rd, 62 KiB/s wr, 30 op/s 2024-05-21T12:38:14.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:14 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:14.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:14 smithi092 ceph-mon[33665]: osdmap e124: 8 total, 8 up, 8 in 2024-05-21T12:38:14.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:14 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:14.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:14 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:14 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:14 smithi045 ceph-mon[30165]: osdmap e124: 8 total, 8 up, 8 in 2024-05-21T12:38:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:14 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:14 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:14 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:14 smithi072 ceph-mon[33912]: osdmap e124: 8 total, 8 up, 8 in 2024-05-21T12:38:14.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:14 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:14.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:14 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:14.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:38:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme2n1 2024-05-21T12:38:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n10 2024-05-21T12:38:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme2n11 2024-05-21T12:38:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme2n12 2024-05-21T12:38:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n13 2024-05-21T12:38:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.2k 0.0k 0.0k 52.0k 0.0k 0.0k nvme2n14 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.8k 0.0k 0.0k 48.0k 0.0k 0.0k nvme2n15 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme2n16 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme2n17 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme2n18 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme2n19 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme2n2 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme2n20 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.8k 0.0k 0.0k 48.0k 0.0k nvme2n3 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme2n4 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme2n5 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme2n6 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.4k 0.0k 0.0k 4.0k 0.0k nvme2n7 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme2n8 2024-05-21T12:38:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n9 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.2k 0.0k 0.0k 12.0k 0.0k nvme5n1 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n10 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme5n11 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme5n12 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme5n13 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme5n14 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme5n15 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme5n16 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme5n17 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.6k 0.0k 0.0k 56.0k 0.0k 0.0k nvme5n18 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme5n19 2024-05-21T12:38:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.4k 0.0k 0.0k 4.0k 0.0k nvme5n2 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme5n20 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.2k 0.0k 0.0k 12.0k 0.0k nvme5n3 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.8k 0.0k 0.0k 48.0k 0.0k 0.0k nvme5n4 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme5n5 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme5n6 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme5n7 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme5n8 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme8n1 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n10 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.2k 0.0k 0.0k 52.0k 0.0k 0.0k nvme8n11 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.2k 0.0k 0.0k 52.0k 0.0k 0.0k nvme8n12 2024-05-21T12:38:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.0k 0.0k 0.0k 60.0k 0.0k 0.0k nvme8n13 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme8n14 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.0k 0.0k 0.0k 60.0k 0.0k nvme8n15 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n16 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme8n17 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.4k 0.0k 0.0k 44.0k 0.0k nvme8n18 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme8n19 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.4k 0.0k 0.0k 44.0k 0.0k nvme8n2 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.8k 0.0k 0.0k 48.0k 0.0k 0.0k nvme8n20 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme8n3 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme8n4 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme8n5 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n6 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.4k 0.0k 0.0k 44.0k 0.0k nvme8n7 2024-05-21T12:38:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:38:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n9 2024-05-21T12:38:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:15 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:15 smithi092 ceph-mon[33665]: osdmap e125: 8 total, 8 up, 8 in 2024-05-21T12:38:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:15 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:15 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:15 smithi092 ceph-mon[33665]: pgmap v743: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 85 KiB/s rd, 71 KiB/s wr, 28 op/s 2024-05-21T12:38:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:15 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:15 smithi045 ceph-mon[30165]: osdmap e125: 8 total, 8 up, 8 in 2024-05-21T12:38:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:15 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:15 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:15.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:15 smithi045 ceph-mon[30165]: pgmap v743: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 85 KiB/s rd, 71 KiB/s wr, 28 op/s 2024-05-21T12:38:15.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:15 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:15.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:15 smithi072 ceph-mon[33912]: osdmap e125: 8 total, 8 up, 8 in 2024-05-21T12:38:15.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:15 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:15.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:15 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:15.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:15 smithi072 ceph-mon[33912]: pgmap v743: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 85 KiB/s rd, 71 KiB/s wr, 28 op/s 2024-05-21T12:38:16.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:16 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:16.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:16 smithi092 ceph-mon[33665]: osdmap e126: 8 total, 8 up, 8 in 2024-05-21T12:38:16.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:16 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:16.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:16 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:16 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:16 smithi045 ceph-mon[30165]: osdmap e126: 8 total, 8 up, 8 in 2024-05-21T12:38:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:16 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:16 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:16 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:16.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:16 smithi072 ceph-mon[33912]: osdmap e126: 8 total, 8 up, 8 in 2024-05-21T12:38:16.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:16 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:16.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:16 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:17.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:17 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:17.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:17 smithi092 ceph-mon[33665]: osdmap e127: 8 total, 8 up, 8 in 2024-05-21T12:38:17.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:17 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:17.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:17 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:17.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:17 smithi092 ceph-mon[33665]: pgmap v746: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-05-21T12:38:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:17 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:17 smithi045 ceph-mon[30165]: osdmap e127: 8 total, 8 up, 8 in 2024-05-21T12:38:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:17 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:17 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:17 smithi045 ceph-mon[30165]: pgmap v746: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-05-21T12:38:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:17 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:17 smithi072 ceph-mon[33912]: osdmap e127: 8 total, 8 up, 8 in 2024-05-21T12:38:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:17 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:17 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:17.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:17 smithi072 ceph-mon[33912]: pgmap v746: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 96 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-05-21T12:38:18.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:18 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:18.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:18 smithi092 ceph-mon[33665]: osdmap e128: 8 total, 8 up, 8 in 2024-05-21T12:38:18.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:18 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:18.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:18 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:18.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:18 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:18.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:18 smithi045 ceph-mon[30165]: osdmap e128: 8 total, 8 up, 8 in 2024-05-21T12:38:18.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:18 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:18.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:18 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:18.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:18 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:18.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:18 smithi072 ceph-mon[33912]: osdmap e128: 8 total, 8 up, 8 in 2024-05-21T12:38:18.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:18 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:18.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:18 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:19.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:19 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:19.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:19 smithi092 ceph-mon[33665]: osdmap e129: 8 total, 8 up, 8 in 2024-05-21T12:38:19.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:19 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:19.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:19 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:19.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:19 smithi092 ceph-mon[33665]: pgmap v749: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 109 KiB/s rd, 118 KiB/s wr, 27 op/s 2024-05-21T12:38:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:19 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:19 smithi045 ceph-mon[30165]: osdmap e129: 8 total, 8 up, 8 in 2024-05-21T12:38:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:19 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:19 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:19 smithi045 ceph-mon[30165]: pgmap v749: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 109 KiB/s rd, 118 KiB/s wr, 27 op/s 2024-05-21T12:38:19.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:19 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:19.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:19 smithi072 ceph-mon[33912]: osdmap e129: 8 total, 8 up, 8 in 2024-05-21T12:38:19.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:19 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:19.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:19 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:19.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:19 smithi072 ceph-mon[33912]: pgmap v749: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 109 KiB/s rd, 118 KiB/s wr, 27 op/s 2024-05-21T12:38:20.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:20 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:20 smithi092 ceph-mon[33665]: osdmap e130: 8 total, 8 up, 8 in 2024-05-21T12:38:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:20 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:20.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:20 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:20 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:20 smithi045 ceph-mon[30165]: osdmap e130: 8 total, 8 up, 8 in 2024-05-21T12:38:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:20 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:20 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:20.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:20 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:20.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:20 smithi072 ceph-mon[33912]: osdmap e130: 8 total, 8 up, 8 in 2024-05-21T12:38:20.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:20 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:20.567 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:20 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:21.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:21 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:21.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:21 smithi092 ceph-mon[33665]: osdmap e131: 8 total, 8 up, 8 in 2024-05-21T12:38:21.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:21 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:21.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:21 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:21.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:21 smithi092 ceph-mon[33665]: pgmap v752: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 94 KiB/s rd, 84 KiB/s wr, 30 op/s 2024-05-21T12:38:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:21 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:21 smithi045 ceph-mon[30165]: osdmap e131: 8 total, 8 up, 8 in 2024-05-21T12:38:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:21 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:21 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:21 smithi045 ceph-mon[30165]: pgmap v752: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 94 KiB/s rd, 84 KiB/s wr, 30 op/s 2024-05-21T12:38:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:21 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:21 smithi072 ceph-mon[33912]: osdmap e131: 8 total, 8 up, 8 in 2024-05-21T12:38:21.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:21 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:21.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:21 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:21.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:21 smithi072 ceph-mon[33912]: pgmap v752: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 94 KiB/s rd, 84 KiB/s wr, 30 op/s 2024-05-21T12:38:22.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:22 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:22.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:22 smithi092 ceph-mon[33665]: osdmap e132: 8 total, 8 up, 8 in 2024-05-21T12:38:22.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:22 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:22.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:22 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:22 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:22 smithi045 ceph-mon[30165]: osdmap e132: 8 total, 8 up, 8 in 2024-05-21T12:38:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:22 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:22 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:22 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:22 smithi072 ceph-mon[33912]: osdmap e132: 8 total, 8 up, 8 in 2024-05-21T12:38:22.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:22 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:22.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:22 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:23.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:23 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:23.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:23 smithi092 ceph-mon[33665]: osdmap e133: 8 total, 8 up, 8 in 2024-05-21T12:38:23.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:23 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:23.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:23 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:23.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:23 smithi092 ceph-mon[33665]: pgmap v755: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-05-21T12:38:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:23 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:23.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:23 smithi045 ceph-mon[30165]: osdmap e133: 8 total, 8 up, 8 in 2024-05-21T12:38:23.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:23 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:23.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:23 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:23.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:23 smithi045 ceph-mon[30165]: pgmap v755: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-05-21T12:38:23.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:23 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:23.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:23 smithi072 ceph-mon[33912]: osdmap e133: 8 total, 8 up, 8 in 2024-05-21T12:38:23.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:23 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:23.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:23 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:23.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:23 smithi072 ceph-mon[33912]: pgmap v755: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 78 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-05-21T12:38:24.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:24 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:24.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:24 smithi045 ceph-mon[30165]: osdmap e134: 8 total, 8 up, 8 in 2024-05-21T12:38:24.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:24 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:24.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:24 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:24.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:24 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:24.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:24 smithi072 ceph-mon[33912]: osdmap e134: 8 total, 8 up, 8 in 2024-05-21T12:38:24.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:24 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:24.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:24 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:24.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:24 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:24.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:24 smithi092 ceph-mon[33665]: osdmap e134: 8 total, 8 up, 8 in 2024-05-21T12:38:24.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:24 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:24.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:24 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:24.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:38:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme2n1 2024-05-21T12:38:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 0.0k 7.2k 0.0k 0.0k 72.0k 0.0k nvme2n10 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 4.4k 4.8k 0.0k 44.0k 48.0k 0.0k nvme2n11 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme2n12 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.4k 0.0k 0.0k 64.0k 0.0k nvme2n13 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme2n14 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme2n15 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme2n16 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme2n17 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme2n18 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n19 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme2n2 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme2n20 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.8k 0.0k 0.0k 28.0k 0.0k nvme2n3 2024-05-21T12:38:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.8k 0.0k 0.0k 48.0k 0.0k 0.0k nvme2n4 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.4k 0.0k 0.0k 64.0k 0.0k 0.0k nvme2n5 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme2n6 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 5.6k 0.0k 0.0k 56.0k 0.0k 0.0k nvme2n7 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme2n8 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 6.4k 0.0k 0.0k 64.0k 0.0k 0.0k nvme2n9 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.6k 0.0k 0.0k 56.0k 0.0k nvme5n1 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 0.8k 0.4k 0.0k 8.0k 4.0k 0.0k nvme5n10 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 0.0k 7.2k 0.0k 0.0k 72.0k 0.0k nvme5n11 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.4k 0.0k 0.0k 44.0k 0.0k 0.0k nvme5n12 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme5n13 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme5n14 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme5n15 2024-05-21T12:38:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.2k 0.0k 0.0k 12.0k 0.0k 0.0k nvme5n16 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.0k 0.0k 0.0k 60.0k 0.0k 0.0k nvme5n17 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme5n18 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.0k 0.0k 0.0k 60.0k 0.0k nvme5n19 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme5n2 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme5n20 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.4k 0.0k 0.0k 64.0k 0.0k nvme5n3 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.8k 0.0k 0.0k 48.0k 0.0k nvme5n4 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme5n5 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme5n6 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.2k 0.0k 0.0k 52.0k 0.0k nvme5n7 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme5n8 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 4.8k 4.0k 0.0k 48.0k 40.0k 0.0k nvme5n9 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme8n1 2024-05-21T12:38:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 0.0k 6.4k 0.0k 0.0k 64.0k 0.0k nvme8n10 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 2.0k 6.0k 0.0k 20.0k 60.0k 0.0k nvme8n11 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.4k 0.0k 0.0k 4.0k 0.0k nvme8n12 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n13 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme8n14 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.8k 0.0k 0.0k 48.0k 0.0k nvme8n15 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.4k 0.0k 0.0k 4.0k 0.0k nvme8n16 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme8n17 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme8n18 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.2k 0.0k 0.0k 12.0k 0.0k nvme8n19 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n2 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme8n20 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.4k 0.0k 0.0k 64.0k 0.0k nvme8n3 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme8n4 2024-05-21T12:38:24.883 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme8n5 2024-05-21T12:38:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme8n6 2024-05-21T12:38:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n7 2024-05-21T12:38:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 2.8k 0.0k 0.0k 28.0k 0.0k 0.0k nvme8n8 2024-05-21T12:38:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 0.20 0.8k 0.8k 0.0k 8.0k 8.0k 0.0k nvme8n9 2024-05-21T12:38:24.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:25.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:25 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:25.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:25 smithi045 ceph-mon[30165]: osdmap e135: 8 total, 8 up, 8 in 2024-05-21T12:38:25.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:25 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:25.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:25 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:25.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:25 smithi045 ceph-mon[30165]: pgmap v758: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 133 KiB/s wr, 19 op/s 2024-05-21T12:38:25.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:25 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:25.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:25 smithi072 ceph-mon[33912]: osdmap e135: 8 total, 8 up, 8 in 2024-05-21T12:38:25.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:25 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:25.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:25 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:25.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:25 smithi072 ceph-mon[33912]: pgmap v758: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 133 KiB/s wr, 19 op/s 2024-05-21T12:38:25.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:25 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:25.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:25 smithi092 ceph-mon[33665]: osdmap e135: 8 total, 8 up, 8 in 2024-05-21T12:38:25.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:25 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:25.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:25 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:25.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:25 smithi092 ceph-mon[33665]: pgmap v758: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 64 KiB/s rd, 133 KiB/s wr, 19 op/s 2024-05-21T12:38:26.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:26 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:26.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:26 smithi045 ceph-mon[30165]: osdmap e136: 8 total, 8 up, 8 in 2024-05-21T12:38:26.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:26 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:26.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:26 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:26 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:26 smithi072 ceph-mon[33912]: osdmap e136: 8 total, 8 up, 8 in 2024-05-21T12:38:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:26 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:26.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:26 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:26.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:26 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:26.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:26 smithi092 ceph-mon[33665]: osdmap e136: 8 total, 8 up, 8 in 2024-05-21T12:38:26.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:26 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:26.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:26 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:27 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:27 smithi045 ceph-mon[30165]: osdmap e137: 8 total, 8 up, 8 in 2024-05-21T12:38:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:27 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:27 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:27 smithi045 ceph-mon[30165]: pgmap v761: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 139 KiB/s rd, 202 KiB/s wr, 32 op/s 2024-05-21T12:38:27.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:27 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:27.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:27 smithi072 ceph-mon[33912]: osdmap e137: 8 total, 8 up, 8 in 2024-05-21T12:38:27.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:27 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:27.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:27 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:27.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:27 smithi072 ceph-mon[33912]: pgmap v761: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 139 KiB/s rd, 202 KiB/s wr, 32 op/s 2024-05-21T12:38:27.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:27 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:27.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:27 smithi092 ceph-mon[33665]: osdmap e137: 8 total, 8 up, 8 in 2024-05-21T12:38:27.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:27 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:27.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:27 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:27.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:27 smithi092 ceph-mon[33665]: pgmap v761: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 139 KiB/s rd, 202 KiB/s wr, 32 op/s 2024-05-21T12:38:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:28 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:28 smithi045 ceph-mon[30165]: osdmap e138: 8 total, 8 up, 8 in 2024-05-21T12:38:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:28 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:28 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:28.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:28 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:28.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:28 smithi072 ceph-mon[33912]: osdmap e138: 8 total, 8 up, 8 in 2024-05-21T12:38:28.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:28 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:28.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:28 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:28.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:28 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:28.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:28 smithi092 ceph-mon[33665]: osdmap e138: 8 total, 8 up, 8 in 2024-05-21T12:38:28.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:28 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:28.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:28 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:29.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:29 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:29.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:29 smithi045 ceph-mon[30165]: osdmap e139: 8 total, 8 up, 8 in 2024-05-21T12:38:29.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:29 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:29.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:29 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:29.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:29 smithi045 ceph-mon[30165]: pgmap v764: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 176 KiB/s rd, 161 KiB/s wr, 34 op/s 2024-05-21T12:38:29.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:29 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:29.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:29 smithi072 ceph-mon[33912]: osdmap e139: 8 total, 8 up, 8 in 2024-05-21T12:38:29.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:29 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:29.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:29 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:29.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:29 smithi072 ceph-mon[33912]: pgmap v764: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 176 KiB/s rd, 161 KiB/s wr, 34 op/s 2024-05-21T12:38:29.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:29 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:29.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:29 smithi092 ceph-mon[33665]: osdmap e139: 8 total, 8 up, 8 in 2024-05-21T12:38:29.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:29 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/2138234012' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:29.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:29 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"} : dispatch 2024-05-21T12:38:29.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:29 smithi092 ceph-mon[33665]: pgmap v764: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 176 KiB/s rd, 161 KiB/s wr, 34 op/s 2024-05-21T12:38:30.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:30 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:30.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:30 smithi045 ceph-mon[30165]: osdmap e140: 8 total, 8 up, 8 in 2024-05-21T12:38:30.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:30 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:30.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:30 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:30 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:30 smithi072 ceph-mon[33912]: osdmap e140: 8 total, 8 up, 8 in 2024-05-21T12:38:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:30 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:30.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:30 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:30.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:30 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/3081013985"}]': finished 2024-05-21T12:38:30.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:30 smithi092 ceph-mon[33665]: osdmap e140: 8 total, 8 up, 8 in 2024-05-21T12:38:30.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:30 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:30.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:30 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:31.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:31 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:31.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:31 smithi045 ceph-mon[30165]: osdmap e141: 8 total, 8 up, 8 in 2024-05-21T12:38:31.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:31 smithi045 ceph-mon[30165]: pgmap v767: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 250 KiB/s rd, 266 KiB/s wr, 31 op/s 2024-05-21T12:38:31.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:31 smithi045 ceph-mon[30165]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:31.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:31 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:31.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:31 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:31.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:31 smithi072 ceph-mon[33912]: osdmap e141: 8 total, 8 up, 8 in 2024-05-21T12:38:31.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:31 smithi072 ceph-mon[33912]: pgmap v767: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 250 KiB/s rd, 266 KiB/s wr, 31 op/s 2024-05-21T12:38:31.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:31 smithi072 ceph-mon[33912]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:31.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:31 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:31.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:31 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:31.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:31 smithi092 ceph-mon[33665]: osdmap e141: 8 total, 8 up, 8 in 2024-05-21T12:38:31.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:31 smithi092 ceph-mon[33665]: pgmap v767: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 250 KiB/s rd, 266 KiB/s wr, 31 op/s 2024-05-21T12:38:31.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:31 smithi092 ceph-mon[33665]: from='client.? 172.21.15.72:0/3603239213' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:31.753 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:31 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"} : dispatch 2024-05-21T12:38:32.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:32 smithi045 ceph-mon[30165]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:32.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:32 smithi045 ceph-mon[30165]: osdmap e142: 8 total, 8 up, 8 in 2024-05-21T12:38:32.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:32 smithi072 ceph-mon[33912]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:32.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:32 smithi072 ceph-mon[33912]: osdmap e142: 8 total, 8 up, 8 in 2024-05-21T12:38:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:32 smithi092 ceph-mon[33665]: from='client.? ' entity='client.nvmeof.nvmeof.b' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.45:0/379124416"}]': finished 2024-05-21T12:38:32.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:32 smithi092 ceph-mon[33665]: osdmap e142: 8 total, 8 up, 8 in 2024-05-21T12:38:33.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:33 smithi045 ceph-mon[30165]: pgmap v769: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-05-21T12:38:33.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:33 smithi072 ceph-mon[33912]: pgmap v769: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-05-21T12:38:33.752 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:33 smithi092 ceph-mon[33665]: pgmap v769: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-05-21T12:38:34.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:38:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 46.0k 66.0k 0.0k 460.0k 660.0k 0.0k nvme2n1 2024-05-21T12:38:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 50.0k 79.2k 0.0k 500.0k 792.0k 0.0k nvme2n10 2024-05-21T12:38:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 48.8k 63.6k 0.0k 488.0k 636.0k 0.0k nvme2n11 2024-05-21T12:38:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 79.6k 45.6k 0.0k 796.0k 456.0k 0.0k nvme2n12 2024-05-21T12:38:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 52.8k 74.8k 0.0k 528.0k 748.0k 0.0k nvme2n13 2024-05-21T12:38:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 71.2k 57.2k 0.0k 712.0k 572.0k 0.0k nvme2n14 2024-05-21T12:38:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 72.4k 52.0k 0.0k 724.0k 520.0k 0.0k nvme2n15 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 60.8k 50.8k 0.0k 608.0k 508.0k 0.0k nvme2n16 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 67.6k 50.4k 0.0k 676.0k 504.0k 0.0k nvme2n17 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 69.2k 40.0k 0.0k 692.0k 400.0k 0.0k nvme2n18 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 63.2k 63.2k 0.0k 632.0k 632.0k 0.0k nvme2n19 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 57.6k 54.8k 0.0k 576.0k 548.0k 0.0k nvme2n2 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 43.6k 62.8k 0.0k 436.0k 628.0k 0.0k nvme2n20 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 53.6k 60.4k 0.0k 536.0k 604.0k 0.0k nvme2n3 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 64.0k 52.0k 0.0k 640.0k 520.0k 0.0k nvme2n4 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 57.2k 81.6k 0.0k 572.0k 816.0k 0.0k nvme2n5 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 50.8k 64.0k 0.0k 508.0k 640.0k 0.0k nvme2n6 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 90.8k 52.0k 0.0k 908.0k 520.0k 0.0k nvme2n7 2024-05-21T12:38:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 60.0k 71.6k 0.0k 600.0k 716.0k 0.0k nvme2n8 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 76.8k 55.6k 0.0k 768.0k 556.0k 0.0k nvme2n9 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 60.0k 61.6k 0.0k 600.0k 616.0k 0.0k nvme5n1 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 60.8k 68.0k 0.0k 608.0k 680.0k 0.0k nvme5n10 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 64.8k 57.2k 0.0k 648.0k 572.0k 0.0k nvme5n11 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 78.4k 56.0k 0.0k 784.0k 560.0k 0.0k nvme5n12 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 75.2k 71.2k 0.0k 752.0k 712.0k 0.0k nvme5n13 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 63.6k 79.6k 0.0k 636.0k 796.0k 0.0k nvme5n14 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 79.2k 69.2k 0.0k 792.0k 692.0k 0.0k nvme5n15 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 44.4k 67.2k 0.0k 444.0k 672.0k 0.0k nvme5n16 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 77.2k 50.0k 0.0k 772.0k 500.0k 0.0k nvme5n17 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 90.0k 56.4k 0.0k 900.0k 564.0k 0.0k nvme5n18 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 69.6k 56.4k 0.0k 696.0k 564.0k 0.0k nvme5n19 2024-05-21T12:38:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 74.0k 56.0k 0.0k 740.0k 560.0k 0.0k nvme5n2 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 58.8k 47.6k 0.0k 588.0k 476.0k 0.0k nvme5n20 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 72.4k 61.6k 0.0k 724.0k 616.0k 0.0k nvme5n3 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 60.4k 61.6k 0.0k 604.0k 616.0k 0.0k nvme5n4 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 54.0k 60.0k 0.0k 540.0k 600.0k 0.0k nvme5n5 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 62.4k 64.4k 0.0k 624.0k 644.0k 0.0k nvme5n6 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 68.4k 72.0k 0.0k 684.0k 720.0k 0.0k nvme5n7 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 46.8k 92.0k 0.0k 468.0k 920.0k 0.0k nvme5n8 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 63.2k 50.0k 0.0k 632.0k 500.0k 0.0k nvme5n9 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 64.0k 58.8k 0.0k 640.0k 588.0k 0.0k nvme8n1 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 60.8k 28.4k 0.0k 608.0k 284.0k 0.0k nvme8n10 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 75.6k 49.6k 0.0k 756.0k 496.0k 0.0k nvme8n11 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 61.2k 54.4k 0.0k 612.0k 544.0k 0.0k nvme8n12 2024-05-21T12:38:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 58.0k 65.2k 0.0k 580.0k 652.0k 0.0k nvme8n13 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 72.0k 56.4k 0.0k 720.0k 564.0k 0.0k nvme8n14 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.20 68.8k 76.0k 0.0k 688.0k 760.0k 0.0k nvme8n15 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 63.2k 54.8k 0.0k 632.0k 548.0k 0.0k nvme8n16 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 41.6k 58.4k 0.0k 416.0k 584.0k 0.0k nvme8n17 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 59.6k 64.8k 0.0k 596.0k 648.0k 0.0k nvme8n18 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 71.2k 39.6k 0.0k 712.0k 396.0k 0.0k nvme8n19 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 44.0k 65.6k 0.0k 440.0k 656.0k 0.0k nvme8n2 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 39.6k 59.2k 0.0k 396.0k 592.0k 0.0k nvme8n20 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 68.4k 62.0k 0.0k 684.0k 620.0k 0.0k nvme8n3 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 80.4k 36.8k 0.0k 804.0k 368.0k 0.0k nvme8n4 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 71.6k 75.2k 0.0k 716.0k 752.0k 0.0k nvme8n5 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 50.0k 69.2k 0.0k 500.0k 692.0k 0.0k nvme8n6 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 53.6k 66.4k 0.0k 536.0k 664.0k 0.0k nvme8n7 2024-05-21T12:38:34.886 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 63.6k 50.0k 0.0k 636.0k 500.0k 0.0k nvme8n8 2024-05-21T12:38:34.887 INFO:tasks.workunit.client.3.smithi137.stdout: 4.10 47.2k 61.2k 0.0k 472.0k 612.0k 0.0k nvme8n9 2024-05-21T12:38:34.887 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:36.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:35 smithi045 ceph-mon[30165]: pgmap v770: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-05-21T12:38:36.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:35 smithi092 ceph-mon[33665]: pgmap v770: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-05-21T12:38:36.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:35 smithi072 ceph-mon[33912]: pgmap v770: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-05-21T12:38:38.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:37 smithi092 ceph-mon[33665]: pgmap v771: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-05-21T12:38:38.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:37 smithi045 ceph-mon[30165]: pgmap v771: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-05-21T12:38:38.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:37 smithi072 ceph-mon[33912]: pgmap v771: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-05-21T12:38:40.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:39 smithi092 ceph-mon[33665]: pgmap v772: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 408 op/s 2024-05-21T12:38:40.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:39 smithi045 ceph-mon[30165]: pgmap v772: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 408 op/s 2024-05-21T12:38:40.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:39 smithi072 ceph-mon[33912]: pgmap v772: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 408 op/s 2024-05-21T12:38:42.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:41 smithi092 ceph-mon[33665]: pgmap v773: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 445 op/s 2024-05-21T12:38:42.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:41 smithi045 ceph-mon[30165]: pgmap v773: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 445 op/s 2024-05-21T12:38:42.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:41 smithi072 ceph-mon[33912]: pgmap v773: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 445 op/s 2024-05-21T12:38:44.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:43 smithi092 ceph-mon[33665]: pgmap v774: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 536 op/s 2024-05-21T12:38:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:43 smithi045 ceph-mon[30165]: pgmap v774: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 536 op/s 2024-05-21T12:38:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:43 smithi072 ceph-mon[33912]: pgmap v774: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 536 op/s 2024-05-21T12:38:44.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:38:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 126.4k 153.6k 0.0k 1.2M 1.5M 0.0k nvme2n1 2024-05-21T12:38:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 121.2k 141.6k 0.0k 1.2M 1.4M 0.0k nvme2n10 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 144.4k 134.4k 0.0k 1.4M 1.3M 0.0k nvme2n11 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 151.6k 140.0k 0.0k 1.5M 1.4M 0.0k nvme2n12 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 101.6k 124.8k 0.0k 1016.0k 1.2M 0.0k nvme2n13 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 131.6k 168.0k 0.0k 1.3M 1.6M 0.0k nvme2n14 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 125.6k 139.6k 0.0k 1.2M 1.4M 0.0k nvme2n15 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 134.4k 127.2k 0.0k 1.3M 1.2M 0.0k nvme2n16 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.90 145.6k 137.2k 0.0k 1.4M 1.3M 0.0k nvme2n17 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 137.6k 122.4k 0.0k 1.3M 1.2M 0.0k nvme2n18 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 150.4k 96.8k 0.0k 1.5M 968.0k 0.0k nvme2n19 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 113.2k 138.0k 0.0k 1.1M 1.3M 0.0k nvme2n2 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 142.4k 116.0k 0.0k 1.4M 1.1M 0.0k nvme2n20 2024-05-21T12:38:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 152.0k 126.8k 0.0k 1.5M 1.2M 0.0k nvme2n3 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 145.2k 118.0k 0.0k 1.4M 1.2M 0.0k nvme2n4 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 148.0k 126.0k 0.0k 1.4M 1.2M 0.0k nvme2n5 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 119.6k 124.8k 0.0k 1.2M 1.2M 0.0k nvme2n6 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 114.4k 144.8k 0.0k 1.1M 1.4M 0.0k nvme2n7 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 126.4k 104.8k 0.0k 1.2M 1.0M 0.0k nvme2n8 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 98.4k 148.4k 0.0k 984.0k 1.4M 0.0k nvme2n9 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 106.0k 136.0k 0.0k 1.0M 1.3M 0.0k nvme5n1 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 128.0k 150.8k 0.0k 1.2M 1.5M 0.0k nvme5n10 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 120.8k 142.0k 0.0k 1.2M 1.4M 0.0k nvme5n11 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 155.2k 160.0k 0.0k 1.5M 1.6M 0.0k nvme5n12 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 136.0k 158.0k 0.0k 1.3M 1.5M 0.0k nvme5n13 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 155.2k 152.8k 0.0k 1.5M 1.5M 0.0k nvme5n14 2024-05-21T12:38:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 134.0k 111.2k 0.0k 1.3M 1.1M 0.0k nvme5n15 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 128.0k 130.0k 0.0k 1.2M 1.3M 0.0k nvme5n16 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.90 135.2k 118.8k 0.0k 1.3M 1.2M 0.0k nvme5n17 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 140.4k 120.8k 0.0k 1.4M 1.2M 0.0k nvme5n18 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 135.2k 126.0k 0.0k 1.3M 1.2M 0.0k nvme5n19 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 138.0k 93.2k 0.0k 1.3M 932.0k 0.0k nvme5n2 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 146.0k 142.0k 0.0k 1.4M 1.4M 0.0k nvme5n20 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 118.8k 177.6k 0.0k 1.2M 1.7M 0.0k nvme5n3 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 133.2k 143.6k 0.0k 1.3M 1.4M 0.0k nvme5n4 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 146.8k 118.4k 0.0k 1.4M 1.2M 0.0k nvme5n5 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 144.4k 116.0k 0.0k 1.4M 1.1M 0.0k nvme5n6 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 131.6k 121.6k 0.0k 1.3M 1.2M 0.0k nvme5n7 2024-05-21T12:38:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 117.2k 158.8k 0.0k 1.1M 1.6M 0.0k nvme5n8 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 112.8k 126.4k 0.0k 1.1M 1.2M 0.0k nvme5n9 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 120.8k 133.2k 0.0k 1.2M 1.3M 0.0k nvme8n1 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 136.0k 114.8k 0.0k 1.3M 1.1M 0.0k nvme8n10 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 142.8k 134.8k 0.0k 1.4M 1.3M 0.0k nvme8n11 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 106.4k 163.6k 0.0k 1.0M 1.6M 0.0k nvme8n12 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 148.8k 128.0k 0.0k 1.5M 1.2M 0.0k nvme8n13 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 151.2k 105.6k 0.0k 1.5M 1.0M 0.0k nvme8n14 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 147.2k 139.2k 0.0k 1.4M 1.4M 0.0k nvme8n15 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.90 133.2k 113.2k 0.0k 1.3M 1.1M 0.0k nvme8n16 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.90 98.8k 178.8k 0.0k 988.0k 1.7M 0.0k nvme8n17 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 156.0k 98.8k 0.0k 1.5M 988.0k 0.0k nvme8n18 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 182.8k 104.0k 0.0k 1.8M 1.0M 0.0k nvme8n19 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 136.0k 138.4k 0.0k 1.3M 1.4M 0.0k nvme8n2 2024-05-21T12:38:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 108.0k 148.8k 0.0k 1.1M 1.5M 0.0k nvme8n20 2024-05-21T12:38:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 138.0k 128.4k 0.0k 1.3M 1.3M 0.0k nvme8n3 2024-05-21T12:38:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 120.4k 126.4k 0.0k 1.2M 1.2M 0.0k nvme8n4 2024-05-21T12:38:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 146.0k 137.6k 0.0k 1.4M 1.3M 0.0k nvme8n5 2024-05-21T12:38:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 147.2k 137.6k 0.0k 1.4M 1.3M 0.0k nvme8n6 2024-05-21T12:38:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 164.4k 123.2k 0.0k 1.6M 1.2M 0.0k nvme8n7 2024-05-21T12:38:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 129.6k 139.6k 0.0k 1.3M 1.4M 0.0k nvme8n8 2024-05-21T12:38:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.80 141.2k 108.0k 0.0k 1.4M 1.1M 0.0k nvme8n9 2024-05-21T12:38:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:46.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:45 smithi092 ceph-mon[33665]: pgmap v775: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 559 op/s 2024-05-21T12:38:46.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:45 smithi045 ceph-mon[30165]: pgmap v775: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 559 op/s 2024-05-21T12:38:46.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:45 smithi072 ceph-mon[33912]: pgmap v775: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 559 op/s 2024-05-21T12:38:48.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:47 smithi092 ceph-mon[33665]: pgmap v776: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.0 MiB/s wr, 601 op/s 2024-05-21T12:38:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:47 smithi045 ceph-mon[30165]: pgmap v776: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.0 MiB/s wr, 601 op/s 2024-05-21T12:38:48.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:47 smithi072 ceph-mon[33912]: pgmap v776: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 8.1 MiB/s rd, 8.0 MiB/s wr, 601 op/s 2024-05-21T12:38:49.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:48 smithi092 ceph-mon[33665]: pgmap v777: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 570 op/s 2024-05-21T12:38:49.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:48 smithi045 ceph-mon[30165]: pgmap v777: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 570 op/s 2024-05-21T12:38:49.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:48 smithi072 ceph-mon[33912]: pgmap v777: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 570 op/s 2024-05-21T12:38:52.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:51 smithi092 ceph-mon[33665]: pgmap v778: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 575 op/s 2024-05-21T12:38:52.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:51 smithi045 ceph-mon[30165]: pgmap v778: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 575 op/s 2024-05-21T12:38:52.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:51 smithi072 ceph-mon[33912]: pgmap v778: 33 pgs: 33 active+clean; 1.0 GiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 575 op/s 2024-05-21T12:38:54.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:53 smithi092 ceph-mon[33665]: pgmap v779: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 637 op/s 2024-05-21T12:38:54.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:53 smithi045 ceph-mon[30165]: pgmap v779: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 637 op/s 2024-05-21T12:38:54.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:53 smithi072 ceph-mon[33912]: pgmap v779: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 637 op/s 2024-05-21T12:38:54.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:38:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 88.8k 146.4k 0.0k 888.0k 1.4M 0.0k nvme2n1 2024-05-21T12:38:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 103.6k 120.8k 0.0k 1.0M 1.2M 0.0k nvme2n10 2024-05-21T12:38:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 118.0k 134.8k 0.0k 1.2M 1.3M 0.0k nvme2n11 2024-05-21T12:38:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 108.8k 145.2k 0.0k 1.1M 1.4M 0.0k nvme2n12 2024-05-21T12:38:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 101.2k 152.8k 0.0k 1012.0k 1.5M 0.0k nvme2n13 2024-05-21T12:38:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 125.6k 142.4k 0.0k 1.2M 1.4M 0.0k nvme2n14 2024-05-21T12:38:54.876 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 121.6k 117.2k 0.0k 1.2M 1.1M 0.0k nvme2n15 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 119.2k 141.2k 0.0k 1.2M 1.4M 0.0k nvme2n16 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 102.0k 130.4k 0.0k 1020.0k 1.3M 0.0k nvme2n17 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 99.2k 107.2k 0.0k 992.0k 1.0M 0.0k nvme2n18 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 111.2k 135.6k 0.0k 1.1M 1.3M 0.0k nvme2n19 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 141.6k 127.2k 0.0k 1.4M 1.2M 0.0k nvme2n2 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 154.8k 80.0k 0.0k 1.5M 800.0k 0.0k nvme2n20 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 106.0k 100.8k 0.0k 1.0M 1008.0k 0.0k nvme2n3 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 98.0k 122.4k 0.0k 980.0k 1.2M 0.0k nvme2n4 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 121.2k 105.6k 0.0k 1.2M 1.0M 0.0k nvme2n5 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 151.6k 97.2k 0.0k 1.5M 972.0k 0.0k nvme2n6 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 96.4k 127.6k 0.0k 964.0k 1.2M 0.0k nvme2n7 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 84.4k 152.4k 0.0k 844.0k 1.5M 0.0k nvme2n8 2024-05-21T12:38:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 93.6k 122.4k 0.0k 936.0k 1.2M 0.0k nvme2n9 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 117.6k 131.2k 0.0k 1.1M 1.3M 0.0k nvme5n1 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 123.6k 96.4k 0.0k 1.2M 964.0k 0.0k nvme5n10 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 105.6k 126.0k 0.0k 1.0M 1.2M 0.0k nvme5n11 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 101.2k 136.0k 0.0k 1012.0k 1.3M 0.0k nvme5n12 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 136.8k 103.2k 0.0k 1.3M 1.0M 0.0k nvme5n13 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 120.0k 137.6k 0.0k 1.2M 1.3M 0.0k nvme5n14 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 94.0k 140.4k 0.0k 940.0k 1.4M 0.0k nvme5n15 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 123.6k 108.4k 0.0k 1.2M 1.1M 0.0k nvme5n16 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 96.8k 118.0k 0.0k 968.0k 1.2M 0.0k nvme5n17 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 121.6k 117.2k 0.0k 1.2M 1.1M 0.0k nvme5n18 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 127.2k 103.6k 0.0k 1.2M 1.0M 0.0k nvme5n19 2024-05-21T12:38:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 125.2k 107.6k 0.0k 1.2M 1.1M 0.0k nvme5n2 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 98.4k 122.0k 0.0k 984.0k 1.2M 0.0k nvme5n20 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 140.8k 96.0k 0.0k 1.4M 960.0k 0.0k nvme5n3 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 133.6k 93.2k 0.0k 1.3M 932.0k 0.0k nvme5n4 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 105.6k 120.4k 0.0k 1.0M 1.2M 0.0k nvme5n5 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 130.4k 112.0k 0.0k 1.3M 1.1M 0.0k nvme5n6 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 112.8k 118.0k 0.0k 1.1M 1.2M 0.0k nvme5n7 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 107.6k 137.2k 0.0k 1.1M 1.3M 0.0k nvme5n8 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 106.8k 96.0k 0.0k 1.0M 960.0k 0.0k nvme5n9 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 103.6k 131.6k 0.0k 1.0M 1.3M 0.0k nvme8n1 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 152.8k 108.8k 0.0k 1.5M 1.1M 0.0k nvme8n10 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 102.4k 108.8k 0.0k 1.0M 1.1M 0.0k nvme8n11 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 120.4k 113.2k 0.0k 1.2M 1.1M 0.0k nvme8n12 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 122.0k 122.4k 0.0k 1.2M 1.2M 0.0k nvme8n13 2024-05-21T12:38:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 102.8k 104.0k 0.0k 1.0M 1.0M 0.0k nvme8n14 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 137.2k 128.4k 0.0k 1.3M 1.3M 0.0k nvme8n15 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 113.6k 102.4k 0.0k 1.1M 1.0M 0.0k nvme8n16 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 112.4k 120.8k 0.0k 1.1M 1.2M 0.0k nvme8n17 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 116.0k 123.6k 0.0k 1.1M 1.2M 0.0k nvme8n18 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 137.6k 96.0k 0.0k 1.3M 960.0k 0.0k nvme8n19 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 124.8k 116.4k 0.0k 1.2M 1.1M 0.0k nvme8n2 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 110.0k 135.2k 0.0k 1.1M 1.3M 0.0k nvme8n20 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 137.6k 122.8k 0.0k 1.3M 1.2M 0.0k nvme8n3 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 153.2k 140.8k 0.0k 1.5M 1.4M 0.0k nvme8n4 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 115.6k 130.4k 0.0k 1.1M 1.3M 0.0k nvme8n5 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 100.8k 124.4k 0.0k 1008.0k 1.2M 0.0k nvme8n6 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 136.0k 91.2k 0.0k 1.3M 912.0k 0.0k nvme8n7 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 106.8k 117.6k 0.0k 1.0M 1.1M 0.0k nvme8n8 2024-05-21T12:38:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.60 145.2k 122.8k 0.0k 1.4M 1.2M 0.0k nvme8n9 2024-05-21T12:38:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:38:56.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:55 smithi092 ceph-mon[33665]: pgmap v780: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 576 op/s 2024-05-21T12:38:56.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:55 smithi045 ceph-mon[30165]: pgmap v780: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 576 op/s 2024-05-21T12:38:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:55 smithi072 ceph-mon[33912]: pgmap v780: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 576 op/s 2024-05-21T12:38:58.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:57 smithi092 ceph-mon[33665]: pgmap v781: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.2 MiB/s wr, 593 op/s 2024-05-21T12:38:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:38:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:57 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:57 smithi045 ceph-mon[30165]: pgmap v781: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.2 MiB/s wr, 593 op/s 2024-05-21T12:38:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:38:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:57 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:57 smithi072 ceph-mon[33912]: pgmap v781: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 7.2 MiB/s wr, 593 op/s 2024-05-21T12:38:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:38:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:57 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-05-21T12:38:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:38:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:38:59.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:59 smithi092 ceph-mon[33665]: pgmap v782: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 550 op/s 2024-05-21T12:38:59.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:59 smithi092 ceph-mon[33665]: pgmap v783: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.7 MiB/s wr, 574 op/s 2024-05-21T12:38:59.504 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:38:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-05-21T12:38:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:38:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:38:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:59 smithi045 ceph-mon[30165]: pgmap v782: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 550 op/s 2024-05-21T12:38:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:59 smithi045 ceph-mon[30165]: pgmap v783: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.7 MiB/s wr, 574 op/s 2024-05-21T12:38:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:38:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:38:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"} : dispatch 2024-05-21T12:38:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:38:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:38:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:59 smithi072 ceph-mon[33912]: pgmap v782: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.5 MiB/s wr, 550 op/s 2024-05-21T12:38:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:59 smithi072 ceph-mon[33912]: pgmap v783: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.8 MiB/s rd, 6.7 MiB/s wr, 574 op/s 2024-05-21T12:38:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:38:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:39:02.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:01 smithi092 ceph-mon[33665]: pgmap v784: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 574 op/s 2024-05-21T12:39:02.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:01 smithi045 ceph-mon[30165]: pgmap v784: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 574 op/s 2024-05-21T12:39:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:01 smithi072 ceph-mon[33912]: pgmap v784: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 574 op/s 2024-05-21T12:39:04.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:03 smithi092 ceph-mon[33665]: pgmap v785: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-05-21T12:39:04.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:03 smithi045 ceph-mon[30165]: pgmap v785: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-05-21T12:39:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:03 smithi072 ceph-mon[33912]: pgmap v785: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-05-21T12:39:04.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 84.0k 124.0k 0.0k 840.0k 1.2M 0.0k nvme2n1 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 82.8k 109.6k 0.0k 828.0k 1.1M 0.0k nvme2n10 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 127.6k 113.2k 0.0k 1.2M 1.1M 0.0k nvme2n11 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 98.8k 143.2k 0.0k 988.0k 1.4M 0.0k nvme2n12 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 91.6k 103.6k 0.0k 916.0k 1.0M 0.0k nvme2n13 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 92.4k 92.8k 0.0k 924.0k 928.0k 0.0k nvme2n14 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 112.8k 107.2k 0.0k 1.1M 1.0M 0.0k nvme2n15 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 104.8k 97.6k 0.0k 1.0M 976.0k 0.0k nvme2n16 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 100.8k 114.4k 0.0k 1008.0k 1.1M 0.0k nvme2n17 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 100.8k 122.0k 0.0k 1008.0k 1.2M 0.0k nvme2n18 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 117.2k 114.8k 0.0k 1.1M 1.1M 0.0k nvme2n19 2024-05-21T12:39:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 99.6k 121.2k 0.0k 996.0k 1.2M 0.0k nvme2n2 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 88.8k 109.2k 0.0k 888.0k 1.1M 0.0k nvme2n20 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 104.8k 106.0k 0.0k 1.0M 1.0M 0.0k nvme2n3 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 96.0k 85.6k 0.0k 960.0k 856.0k 0.0k nvme2n4 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 79.6k 116.4k 0.0k 796.0k 1.1M 0.0k nvme2n5 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 104.4k 115.2k 0.0k 1.0M 1.1M 0.0k nvme2n6 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 84.8k 116.0k 0.0k 848.0k 1.1M 0.0k nvme2n7 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 105.6k 98.4k 0.0k 1.0M 984.0k 0.0k nvme2n8 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 144.4k 74.0k 0.0k 1.4M 740.0k 0.0k nvme2n9 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 88.0k 118.8k 0.0k 880.0k 1.2M 0.0k nvme5n1 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 100.0k 114.8k 0.0k 1000.0k 1.1M 0.0k nvme5n10 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 102.8k 103.6k 0.0k 1.0M 1.0M 0.0k nvme5n11 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 112.4k 100.0k 0.0k 1.1M 1000.0k 0.0k nvme5n12 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 68.0k 133.6k 0.0k 680.0k 1.3M 0.0k nvme5n13 2024-05-21T12:39:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 126.8k 91.2k 0.0k 1.2M 912.0k 0.0k nvme5n14 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 109.6k 98.0k 0.0k 1.1M 980.0k 0.0k nvme5n15 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 92.0k 90.4k 0.0k 920.0k 904.0k 0.0k nvme5n16 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 110.8k 103.6k 0.0k 1.1M 1.0M 0.0k nvme5n17 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 65.2k 122.4k 0.0k 652.0k 1.2M 0.0k nvme5n18 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 93.2k 120.8k 0.0k 932.0k 1.2M 0.0k nvme5n19 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 104.8k 126.4k 0.0k 1.0M 1.2M 0.0k nvme5n2 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 137.6k 88.0k 0.0k 1.3M 880.0k 0.0k nvme5n20 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 106.0k 110.0k 0.0k 1.0M 1.1M 0.0k nvme5n3 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 127.2k 108.8k 0.0k 1.2M 1.1M 0.0k nvme5n4 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 106.8k 115.2k 0.0k 1.0M 1.1M 0.0k nvme5n5 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 114.8k 101.6k 0.0k 1.1M 1016.0k 0.0k nvme5n6 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 98.8k 118.8k 0.0k 988.0k 1.2M 0.0k nvme5n7 2024-05-21T12:39:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 105.2k 119.6k 0.0k 1.0M 1.2M 0.0k nvme5n8 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 99.2k 105.6k 0.0k 992.0k 1.0M 0.0k nvme5n9 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 121.6k 84.4k 0.0k 1.2M 844.0k 0.0k nvme8n1 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 102.0k 102.8k 0.0k 1020.0k 1.0M 0.0k nvme8n10 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 95.6k 110.4k 0.0k 956.0k 1.1M 0.0k nvme8n11 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 72.4k 112.0k 0.0k 724.0k 1.1M 0.0k nvme8n12 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 117.6k 103.6k 0.0k 1.1M 1.0M 0.0k nvme8n13 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 94.0k 104.8k 0.0k 940.0k 1.0M 0.0k nvme8n14 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 107.6k 96.0k 0.0k 1.1M 960.0k 0.0k nvme8n15 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 91.6k 94.0k 0.0k 916.0k 940.0k 0.0k nvme8n16 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 123.6k 106.0k 0.0k 1.2M 1.0M 0.0k nvme8n17 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 108.4k 113.6k 0.0k 1.1M 1.1M 0.0k nvme8n18 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 90.0k 112.8k 0.0k 900.0k 1.1M 0.0k nvme8n19 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 98.8k 123.2k 0.0k 988.0k 1.2M 0.0k nvme8n2 2024-05-21T12:39:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 104.4k 100.8k 0.0k 1.0M 1008.0k 0.0k nvme8n20 2024-05-21T12:39:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 104.8k 103.2k 0.0k 1.0M 1.0M 0.0k nvme8n3 2024-05-21T12:39:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 118.8k 100.0k 0.0k 1.2M 1000.0k 0.0k nvme8n4 2024-05-21T12:39:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 131.2k 114.4k 0.0k 1.3M 1.1M 0.0k nvme8n5 2024-05-21T12:39:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 101.6k 101.2k 0.0k 1016.0k 1012.0k 0.0k nvme8n6 2024-05-21T12:39:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 99.2k 141.6k 0.0k 992.0k 1.4M 0.0k nvme8n7 2024-05-21T12:39:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 94.0k 110.8k 0.0k 940.0k 1.1M 0.0k nvme8n8 2024-05-21T12:39:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 90.0k 117.6k 0.0k 900.0k 1.1M 0.0k nvme8n9 2024-05-21T12:39:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:06.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:05 smithi092 ceph-mon[33665]: pgmap v786: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 529 op/s 2024-05-21T12:39:06.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:05 smithi045 ceph-mon[30165]: pgmap v786: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 529 op/s 2024-05-21T12:39:06.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:05 smithi072 ceph-mon[33912]: pgmap v786: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 529 op/s 2024-05-21T12:39:08.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:07 smithi092 ceph-mon[33665]: pgmap v787: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 568 op/s 2024-05-21T12:39:08.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:07 smithi045 ceph-mon[30165]: pgmap v787: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 568 op/s 2024-05-21T12:39:08.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:07 smithi072 ceph-mon[33912]: pgmap v787: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 568 op/s 2024-05-21T12:39:10.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:09 smithi092 ceph-mon[33665]: pgmap v788: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 566 op/s 2024-05-21T12:39:10.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:09 smithi045 ceph-mon[30165]: pgmap v788: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 566 op/s 2024-05-21T12:39:10.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:09 smithi072 ceph-mon[33912]: pgmap v788: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 566 op/s 2024-05-21T12:39:12.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:11 smithi092 ceph-mon[33665]: pgmap v789: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 551 op/s 2024-05-21T12:39:12.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:11 smithi045 ceph-mon[30165]: pgmap v789: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 551 op/s 2024-05-21T12:39:12.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:11 smithi072 ceph-mon[33912]: pgmap v789: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 551 op/s 2024-05-21T12:39:14.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:13 smithi092 ceph-mon[33665]: pgmap v790: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-21T12:39:14.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:13 smithi045 ceph-mon[30165]: pgmap v790: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-21T12:39:14.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:13 smithi072 ceph-mon[33912]: pgmap v790: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-05-21T12:39:14.867 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 95.6k 83.2k 0.0k 956.0k 832.0k 0.0k nvme2n1 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 100.8k 93.2k 0.0k 1008.0k 932.0k 0.0k nvme2n10 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 102.8k 104.8k 0.0k 1.0M 1.0M 0.0k nvme2n11 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 98.4k 118.4k 0.0k 984.0k 1.2M 0.0k nvme2n12 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 96.0k 112.8k 0.0k 960.0k 1.1M 0.0k nvme2n13 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 111.2k 116.0k 0.0k 1.1M 1.1M 0.0k nvme2n14 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 113.6k 94.4k 0.0k 1.1M 944.0k 0.0k nvme2n15 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 112.0k 90.8k 0.0k 1.1M 908.0k 0.0k nvme2n16 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 111.2k 90.0k 0.0k 1.1M 900.0k 0.0k nvme2n17 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 128.4k 98.4k 0.0k 1.3M 984.0k 0.0k nvme2n18 2024-05-21T12:39:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 118.8k 89.6k 0.0k 1.2M 896.0k 0.0k nvme2n19 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 100.4k 92.8k 0.0k 1004.0k 928.0k 0.0k nvme2n2 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 91.2k 92.4k 0.0k 912.0k 924.0k 0.0k nvme2n20 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 92.0k 101.2k 0.0k 920.0k 1012.0k 0.0k nvme2n3 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 97.2k 105.6k 0.0k 972.0k 1.0M 0.0k nvme2n4 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 110.4k 84.0k 0.0k 1.1M 840.0k 0.0k nvme2n5 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 98.8k 73.2k 0.0k 988.0k 732.0k 0.0k nvme2n6 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 87.2k 115.2k 0.0k 872.0k 1.1M 0.0k nvme2n7 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 122.8k 71.2k 0.0k 1.2M 712.0k 0.0k nvme2n8 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 101.6k 71.2k 0.0k 1016.0k 712.0k 0.0k nvme2n9 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 90.4k 94.8k 0.0k 904.0k 948.0k 0.0k nvme5n1 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 78.4k 88.8k 0.0k 784.0k 888.0k 0.0k nvme5n10 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 109.2k 102.8k 0.0k 1.1M 1.0M 0.0k nvme5n11 2024-05-21T12:39:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 91.2k 111.2k 0.0k 912.0k 1.1M 0.0k nvme5n12 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 103.2k 82.4k 0.0k 1.0M 824.0k 0.0k nvme5n13 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 110.0k 124.0k 0.0k 1.1M 1.2M 0.0k nvme5n14 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 109.6k 99.6k 0.0k 1.1M 996.0k 0.0k nvme5n15 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 87.2k 119.2k 0.0k 872.0k 1.2M 0.0k nvme5n16 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 77.6k 117.6k 0.0k 776.0k 1.1M 0.0k nvme5n17 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 119.6k 91.6k 0.0k 1.2M 916.0k 0.0k nvme5n18 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 122.8k 93.6k 0.0k 1.2M 936.0k 0.0k nvme5n19 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 132.4k 79.6k 0.0k 1.3M 796.0k 0.0k nvme5n2 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 104.8k 114.4k 0.0k 1.0M 1.1M 0.0k nvme5n20 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 99.6k 108.4k 0.0k 996.0k 1.1M 0.0k nvme5n3 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 105.6k 136.8k 0.0k 1.0M 1.3M 0.0k nvme5n4 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 108.4k 96.8k 0.0k 1.1M 968.0k 0.0k nvme5n5 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 112.0k 96.0k 0.0k 1.1M 960.0k 0.0k nvme5n6 2024-05-21T12:39:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 95.2k 91.2k 0.0k 952.0k 912.0k 0.0k nvme5n7 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 128.0k 81.6k 0.0k 1.2M 816.0k 0.0k nvme5n8 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 79.6k 129.2k 0.0k 796.0k 1.3M 0.0k nvme5n9 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 97.6k 106.4k 0.0k 976.0k 1.0M 0.0k nvme8n1 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 86.4k 107.6k 0.0k 864.0k 1.1M 0.0k nvme8n10 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 86.8k 90.4k 0.0k 868.0k 904.0k 0.0k nvme8n11 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 111.6k 86.4k 0.0k 1.1M 864.0k 0.0k nvme8n12 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 112.8k 67.2k 0.0k 1.1M 672.0k 0.0k nvme8n13 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 70.4k 86.0k 0.0k 704.0k 860.0k 0.0k nvme8n14 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 94.8k 128.8k 0.0k 948.0k 1.3M 0.0k nvme8n15 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 120.8k 83.2k 0.0k 1.2M 832.0k 0.0k nvme8n16 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 127.2k 71.2k 0.0k 1.2M 712.0k 0.0k nvme8n17 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 112.0k 82.4k 0.0k 1.1M 824.0k 0.0k nvme8n18 2024-05-21T12:39:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 97.6k 96.0k 0.0k 976.0k 960.0k 0.0k nvme8n19 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 108.8k 103.2k 0.0k 1.1M 1.0M 0.0k nvme8n2 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 102.0k 76.4k 0.0k 1020.0k 764.0k 0.0k nvme8n20 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 87.6k 114.8k 0.0k 876.0k 1.1M 0.0k nvme8n3 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 146.8k 85.2k 0.0k 1.4M 852.0k 0.0k nvme8n4 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.50 88.0k 89.2k 0.0k 880.0k 892.0k 0.0k nvme8n5 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 86.0k 116.4k 0.0k 860.0k 1.1M 0.0k nvme8n6 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 68.0k 118.4k 0.0k 680.0k 1.2M 0.0k nvme8n7 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 98.4k 71.2k 0.0k 984.0k 712.0k 0.0k nvme8n8 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.40 113.2k 108.0k 0.0k 1.1M 1.1M 0.0k nvme8n9 2024-05-21T12:39:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:16.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:15 smithi092 ceph-mon[33665]: pgmap v791: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 520 op/s 2024-05-21T12:39:16.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:15 smithi045 ceph-mon[30165]: pgmap v791: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 520 op/s 2024-05-21T12:39:16.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:15 smithi072 ceph-mon[33912]: pgmap v791: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 520 op/s 2024-05-21T12:39:18.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:17 smithi092 ceph-mon[33665]: pgmap v792: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 613 op/s 2024-05-21T12:39:18.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:17 smithi045 ceph-mon[30165]: pgmap v792: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 613 op/s 2024-05-21T12:39:18.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:17 smithi072 ceph-mon[33912]: pgmap v792: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 613 op/s 2024-05-21T12:39:20.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:19 smithi092 ceph-mon[33665]: pgmap v793: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2024-05-21T12:39:20.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:19 smithi045 ceph-mon[30165]: pgmap v793: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2024-05-21T12:39:20.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:19 smithi072 ceph-mon[33912]: pgmap v793: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2024-05-21T12:39:22.217 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:21 smithi072 ceph-mon[33912]: pgmap v794: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 556 op/s 2024-05-21T12:39:22.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:21 smithi092 ceph-mon[33665]: pgmap v794: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 556 op/s 2024-05-21T12:39:22.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:21 smithi045 ceph-mon[30165]: pgmap v794: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 556 op/s 2024-05-21T12:39:24.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:23 smithi092 ceph-mon[33665]: pgmap v795: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 616 op/s 2024-05-21T12:39:24.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:23 smithi045 ceph-mon[30165]: pgmap v795: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 616 op/s 2024-05-21T12:39:24.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:23 smithi072 ceph-mon[33912]: pgmap v795: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 616 op/s 2024-05-21T12:39:24.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:39:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 86.4k 89.2k 0.0k 864.0k 892.0k 0.0k nvme2n1 2024-05-21T12:39:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 84.0k 104.4k 0.0k 840.0k 1.0M 0.0k nvme2n10 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 84.0k 90.8k 0.0k 840.0k 908.0k 0.0k nvme2n11 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 118.8k 72.4k 0.0k 1.2M 724.0k 0.0k nvme2n12 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 112.8k 100.4k 0.0k 1.1M 1004.0k 0.0k nvme2n13 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 96.0k 88.8k 0.0k 960.0k 888.0k 0.0k nvme2n14 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 79.6k 92.0k 0.0k 796.0k 920.0k 0.0k nvme2n15 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 82.0k 94.0k 0.0k 820.0k 940.0k 0.0k nvme2n16 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 82.8k 101.6k 0.0k 828.0k 1016.0k 0.0k nvme2n17 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 96.4k 100.8k 0.0k 964.0k 1008.0k 0.0k nvme2n18 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 84.8k 105.6k 0.0k 848.0k 1.0M 0.0k nvme2n19 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 98.8k 88.0k 0.0k 988.0k 880.0k 0.0k nvme2n2 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 73.2k 74.0k 0.0k 732.0k 740.0k 0.0k nvme2n20 2024-05-21T12:39:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 83.6k 106.8k 0.0k 836.0k 1.0M 0.0k nvme2n3 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 82.8k 99.2k 0.0k 828.0k 992.0k 0.0k nvme2n4 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 79.2k 105.6k 0.0k 792.0k 1.0M 0.0k nvme2n5 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 88.4k 79.2k 0.0k 884.0k 792.0k 0.0k nvme2n6 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 82.0k 95.2k 0.0k 820.0k 952.0k 0.0k nvme2n7 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 79.2k 86.8k 0.0k 792.0k 868.0k 0.0k nvme2n8 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 94.8k 88.8k 0.0k 948.0k 888.0k 0.0k nvme2n9 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 91.2k 84.8k 0.0k 912.0k 848.0k 0.0k nvme5n1 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 96.0k 85.6k 0.0k 960.0k 856.0k 0.0k nvme5n10 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 76.0k 108.4k 0.0k 760.0k 1.1M 0.0k nvme5n11 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 103.2k 84.8k 0.0k 1.0M 848.0k 0.0k nvme5n12 2024-05-21T12:39:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 86.0k 107.2k 0.0k 860.0k 1.0M 0.0k nvme5n13 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 92.4k 122.4k 0.0k 924.0k 1.2M 0.0k nvme5n14 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 100.4k 90.8k 0.0k 1004.0k 908.0k 0.0k nvme5n15 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 90.4k 84.4k 0.0k 904.0k 844.0k 0.0k nvme5n16 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 100.4k 95.2k 0.0k 1004.0k 952.0k 0.0k nvme5n17 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 63.2k 116.8k 0.0k 632.0k 1.1M 0.0k nvme5n18 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 82.0k 74.8k 0.0k 820.0k 748.0k 0.0k nvme5n19 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 92.8k 86.8k 0.0k 928.0k 868.0k 0.0k nvme5n2 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 93.2k 102.0k 0.0k 932.0k 1020.0k 0.0k nvme5n20 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 104.0k 98.4k 0.0k 1.0M 984.0k 0.0k nvme5n3 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 97.2k 108.4k 0.0k 972.0k 1.1M 0.0k nvme5n4 2024-05-21T12:39:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 60.8k 111.2k 0.0k 608.0k 1.1M 0.0k nvme5n5 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 100.0k 84.4k 0.0k 1000.0k 844.0k 0.0k nvme5n6 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 108.8k 78.4k 0.0k 1.1M 784.0k 0.0k nvme5n7 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 103.6k 96.4k 0.0k 1.0M 964.0k 0.0k nvme5n8 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 96.4k 74.0k 0.0k 964.0k 740.0k 0.0k nvme5n9 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 72.4k 96.8k 0.0k 724.0k 968.0k 0.0k nvme8n1 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 83.2k 89.6k 0.0k 832.0k 896.0k 0.0k nvme8n10 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 98.4k 86.8k 0.0k 984.0k 868.0k 0.0k nvme8n11 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 96.8k 99.6k 0.0k 968.0k 996.0k 0.0k nvme8n12 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 87.2k 105.2k 0.0k 872.0k 1.0M 0.0k nvme8n13 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 97.6k 89.6k 0.0k 976.0k 896.0k 0.0k nvme8n14 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 86.8k 89.2k 0.0k 868.0k 892.0k 0.0k nvme8n15 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 114.0k 65.2k 0.0k 1.1M 652.0k 0.0k nvme8n16 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 82.8k 94.8k 0.0k 828.0k 948.0k 0.0k nvme8n17 2024-05-21T12:39:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 89.6k 99.2k 0.0k 896.0k 992.0k 0.0k nvme8n18 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 88.8k 120.4k 0.0k 888.0k 1.2M 0.0k nvme8n19 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 102.0k 77.2k 0.0k 1020.0k 772.0k 0.0k nvme8n2 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 72.8k 87.6k 0.0k 728.0k 876.0k 0.0k nvme8n20 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 99.6k 110.4k 0.0k 996.0k 1.1M 0.0k nvme8n3 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 76.4k 100.8k 0.0k 764.0k 1008.0k 0.0k nvme8n4 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 79.6k 83.2k 0.0k 796.0k 832.0k 0.0k nvme8n5 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 88.8k 89.6k 0.0k 888.0k 896.0k 0.0k nvme8n6 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 99.6k 72.8k 0.0k 996.0k 728.0k 0.0k nvme8n7 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 76.8k 92.0k 0.0k 768.0k 920.0k 0.0k nvme8n8 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.30 88.8k 77.6k 0.0k 888.0k 776.0k 0.0k nvme8n9 2024-05-21T12:39:24.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:25.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:24 smithi092 ceph-mon[33665]: pgmap v796: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 522 op/s 2024-05-21T12:39:25.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:25 smithi045 ceph-mon[30165]: pgmap v796: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 522 op/s 2024-05-21T12:39:25.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:25 smithi072 ceph-mon[33912]: pgmap v796: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 522 op/s 2024-05-21T12:39:28.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:27 smithi092 ceph-mon[33665]: pgmap v797: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-05-21T12:39:28.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:27 smithi045 ceph-mon[30165]: pgmap v797: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-05-21T12:39:28.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:27 smithi072 ceph-mon[33912]: pgmap v797: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 610 op/s 2024-05-21T12:39:30.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:29 smithi092 ceph-mon[33665]: pgmap v798: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2024-05-21T12:39:30.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:29 smithi045 ceph-mon[30165]: pgmap v798: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2024-05-21T12:39:30.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:29 smithi072 ceph-mon[33912]: pgmap v798: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 533 op/s 2024-05-21T12:39:32.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:31 smithi092 ceph-mon[33665]: pgmap v799: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-05-21T12:39:32.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:31 smithi045 ceph-mon[30165]: pgmap v799: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-05-21T12:39:32.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:31 smithi072 ceph-mon[33912]: pgmap v799: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-05-21T12:39:34.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:33 smithi092 ceph-mon[33665]: pgmap v800: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-05-21T12:39:34.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:33 smithi045 ceph-mon[30165]: pgmap v800: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-05-21T12:39:34.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:33 smithi072 ceph-mon[33912]: pgmap v800: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 611 op/s 2024-05-21T12:39:34.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 141.6k 121.2k 0.0k 1.4M 1.2M 0.0k nvme2n1 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 128.0k 109.6k 0.0k 1.2M 1.1M 0.0k nvme2n10 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 87.2k 103.6k 0.0k 872.0k 1.0M 0.0k nvme2n11 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 79.6k 104.8k 0.0k 796.0k 1.0M 0.0k nvme2n12 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 108.0k 157.6k 0.0k 1.1M 1.5M 0.0k nvme2n13 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 106.8k 71.6k 0.0k 1.0M 716.0k 0.0k nvme2n14 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 123.2k 95.2k 0.0k 1.2M 952.0k 0.0k nvme2n15 2024-05-21T12:39:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 79.2k 78.4k 0.0k 792.0k 784.0k 0.0k nvme2n16 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 120.4k 60.8k 0.0k 1.2M 608.0k 0.0k nvme2n17 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 102.8k 140.0k 0.0k 1.0M 1.4M 0.0k nvme2n18 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 95.2k 96.8k 0.0k 952.0k 968.0k 0.0k nvme2n19 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 89.6k 90.8k 0.0k 896.0k 908.0k 0.0k nvme2n2 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 159.6k 149.6k 0.0k 1.6M 1.5M 0.0k nvme2n20 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 93.2k 99.2k 0.0k 932.0k 992.0k 0.0k nvme2n3 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 124.0k 131.2k 0.0k 1.2M 1.3M 0.0k nvme2n4 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 84.0k 66.8k 0.0k 840.0k 668.0k 0.0k nvme2n5 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 150.8k 134.0k 0.0k 1.5M 1.3M 0.0k nvme2n6 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 152.4k 132.0k 0.0k 1.5M 1.3M 0.0k nvme2n7 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 66.0k 77.6k 0.0k 660.0k 776.0k 0.0k nvme2n8 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 74.0k 86.0k 0.0k 740.0k 860.0k 0.0k nvme2n9 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 121.2k 128.8k 0.0k 1.2M 1.3M 0.0k nvme5n1 2024-05-21T12:39:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 94.4k 86.0k 0.0k 944.0k 860.0k 0.0k nvme5n10 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 106.4k 90.4k 0.0k 1.0M 904.0k 0.0k nvme5n11 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 81.6k 103.6k 0.0k 816.0k 1.0M 0.0k nvme5n12 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 108.8k 80.0k 0.0k 1.1M 800.0k 0.0k nvme5n13 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 98.4k 89.6k 0.0k 984.0k 896.0k 0.0k nvme5n14 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 96.8k 92.8k 0.0k 968.0k 928.0k 0.0k nvme5n15 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 91.6k 111.6k 0.0k 916.0k 1.1M 0.0k nvme5n16 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 136.8k 126.4k 0.0k 1.3M 1.2M 0.0k nvme5n17 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 93.6k 77.2k 0.0k 936.0k 772.0k 0.0k nvme5n18 2024-05-21T12:39:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 83.2k 92.8k 0.0k 832.0k 928.0k 0.0k nvme5n19 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 116.0k 137.6k 0.0k 1.1M 1.3M 0.0k nvme5n2 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 76.4k 102.0k 0.0k 764.0k 1020.0k 0.0k nvme5n20 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 106.0k 69.6k 0.0k 1.0M 696.0k 0.0k nvme5n3 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 80.0k 102.8k 0.0k 800.0k 1.0M 0.0k nvme5n4 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 126.0k 83.2k 0.0k 1.2M 832.0k 0.0k nvme5n5 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 90.0k 84.8k 0.0k 900.0k 848.0k 0.0k nvme5n6 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 124.0k 131.6k 0.0k 1.2M 1.3M 0.0k nvme5n7 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 92.4k 73.6k 0.0k 924.0k 736.0k 0.0k nvme5n8 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 112.4k 99.2k 0.0k 1.1M 992.0k 0.0k nvme5n9 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 87.2k 58.0k 0.0k 872.0k 580.0k 0.0k nvme8n1 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 99.6k 89.6k 0.0k 996.0k 896.0k 0.0k nvme8n10 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 68.4k 84.0k 0.0k 684.0k 840.0k 0.0k nvme8n11 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 80.8k 122.8k 0.0k 808.0k 1.2M 0.0k nvme8n12 2024-05-21T12:39:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 105.2k 91.6k 0.0k 1.0M 916.0k 0.0k nvme8n13 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 140.8k 118.4k 0.0k 1.4M 1.2M 0.0k nvme8n14 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 93.2k 96.0k 0.0k 932.0k 960.0k 0.0k nvme8n15 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 156.4k 117.2k 0.0k 1.5M 1.1M 0.0k nvme8n16 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 108.4k 72.4k 0.0k 1.1M 724.0k 0.0k nvme8n17 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 77.6k 89.2k 0.0k 776.0k 892.0k 0.0k nvme8n18 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 88.4k 81.6k 0.0k 884.0k 816.0k 0.0k nvme8n19 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 111.6k 85.6k 0.0k 1.1M 856.0k 0.0k nvme8n2 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 90.0k 79.6k 0.0k 900.0k 796.0k 0.0k nvme8n20 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 109.6k 81.6k 0.0k 1.1M 816.0k 0.0k nvme8n3 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 56.8k 104.0k 0.0k 568.0k 1.0M 0.0k nvme8n4 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 102.8k 59.2k 0.0k 1.0M 592.0k 0.0k nvme8n5 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 94.8k 83.2k 0.0k 948.0k 832.0k 0.0k nvme8n6 2024-05-21T12:39:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 104.4k 90.8k 0.0k 1.0M 908.0k 0.0k nvme8n7 2024-05-21T12:39:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 98.0k 108.4k 0.0k 980.0k 1.1M 0.0k nvme8n8 2024-05-21T12:39:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.20 86.0k 80.4k 0.0k 860.0k 804.0k 0.0k nvme8n9 2024-05-21T12:39:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:36.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:35 smithi092 ceph-mon[33665]: pgmap v801: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-05-21T12:39:36.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:35 smithi045 ceph-mon[30165]: pgmap v801: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-05-21T12:39:36.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:35 smithi072 ceph-mon[33912]: pgmap v801: 33 pgs: 33 active+clean; 1.0 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-05-21T12:39:38.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:37 smithi092 ceph-mon[33665]: pgmap v802: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-05-21T12:39:38.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:37 smithi045 ceph-mon[30165]: pgmap v802: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-05-21T12:39:38.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:37 smithi072 ceph-mon[33912]: pgmap v802: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-05-21T12:39:40.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:39 smithi092 ceph-mon[33665]: pgmap v803: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.6 MiB/s wr, 537 op/s 2024-05-21T12:39:40.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:39 smithi045 ceph-mon[30165]: pgmap v803: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.6 MiB/s wr, 537 op/s 2024-05-21T12:39:40.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:39 smithi072 ceph-mon[33912]: pgmap v803: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.6 MiB/s wr, 537 op/s 2024-05-21T12:39:42.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:41 smithi092 ceph-mon[33665]: pgmap v804: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.0 MiB/s wr, 551 op/s 2024-05-21T12:39:42.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:41 smithi045 ceph-mon[30165]: pgmap v804: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.0 MiB/s wr, 551 op/s 2024-05-21T12:39:42.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:41 smithi072 ceph-mon[33912]: pgmap v804: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.0 MiB/s wr, 551 op/s 2024-05-21T12:39:44.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:43 smithi092 ceph-mon[33665]: pgmap v805: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.0 MiB/s wr, 601 op/s 2024-05-21T12:39:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:43 smithi045 ceph-mon[30165]: pgmap v805: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.0 MiB/s wr, 601 op/s 2024-05-21T12:39:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:43 smithi072 ceph-mon[33912]: pgmap v805: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.0 MiB/s wr, 601 op/s 2024-05-21T12:39:44.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:39:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 123.6k 183.6k 0.0k 1.2M 1.8M 0.0k nvme2n1 2024-05-21T12:39:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 136.8k 123.2k 0.0k 1.3M 1.2M 0.0k nvme2n10 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 74.0k 67.2k 0.0k 740.0k 672.0k 0.0k nvme2n11 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 100.4k 82.4k 0.0k 1004.0k 824.0k 0.0k nvme2n12 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 171.6k 144.8k 0.0k 1.7M 1.4M 0.0k nvme2n13 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 117.6k 131.2k 0.0k 1.1M 1.3M 0.0k nvme2n14 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 120.8k 161.6k 0.0k 1.2M 1.6M 0.0k nvme2n15 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 130.8k 126.4k 0.0k 1.3M 1.2M 0.0k nvme2n16 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 144.4k 114.0k 0.0k 1.4M 1.1M 0.0k nvme2n17 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 146.0k 116.0k 0.0k 1.4M 1.1M 0.0k nvme2n18 2024-05-21T12:39:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 161.2k 133.2k 0.0k 1.6M 1.3M 0.0k nvme2n19 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 163.2k 124.4k 0.0k 1.6M 1.2M 0.0k nvme2n2 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 148.0k 153.2k 0.0k 1.4M 1.5M 0.0k nvme2n20 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 135.2k 170.4k 0.0k 1.3M 1.7M 0.0k nvme2n3 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 129.6k 115.2k 0.0k 1.3M 1.1M 0.0k nvme2n4 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 156.0k 128.0k 0.0k 1.5M 1.2M 0.0k nvme2n5 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 124.4k 165.2k 0.0k 1.2M 1.6M 0.0k nvme2n6 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 151.6k 140.4k 0.0k 1.5M 1.4M 0.0k nvme2n7 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 121.6k 141.2k 0.0k 1.2M 1.4M 0.0k nvme2n8 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 173.6k 116.0k 0.0k 1.7M 1.1M 0.0k nvme2n9 2024-05-21T12:39:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 110.0k 152.4k 0.0k 1.1M 1.5M 0.0k nvme5n1 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 94.4k 106.0k 0.0k 944.0k 1.0M 0.0k nvme5n10 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 165.6k 133.2k 0.0k 1.6M 1.3M 0.0k nvme5n11 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 106.8k 92.4k 0.0k 1.0M 924.0k 0.0k nvme5n12 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 144.0k 100.8k 0.0k 1.4M 1008.0k 0.0k nvme5n13 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 87.2k 76.8k 0.0k 872.0k 768.0k 0.0k nvme5n14 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 90.8k 104.8k 0.0k 908.0k 1.0M 0.0k nvme5n15 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 173.6k 110.8k 0.0k 1.7M 1.1M 0.0k nvme5n16 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 99.2k 157.6k 0.0k 992.0k 1.5M 0.0k nvme5n17 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 125.6k 128.8k 0.0k 1.2M 1.3M 0.0k nvme5n18 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 134.4k 114.4k 0.0k 1.3M 1.1M 0.0k nvme5n19 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 154.4k 120.8k 0.0k 1.5M 1.2M 0.0k nvme5n2 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 149.6k 115.2k 0.0k 1.5M 1.1M 0.0k nvme5n20 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 98.4k 114.8k 0.0k 984.0k 1.1M 0.0k nvme5n3 2024-05-21T12:39:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 93.2k 109.2k 0.0k 932.0k 1.1M 0.0k nvme5n4 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 146.4k 146.8k 0.0k 1.4M 1.4M 0.0k nvme5n5 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 88.0k 126.8k 0.0k 880.0k 1.2M 0.0k nvme5n6 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 147.2k 147.2k 0.0k 1.4M 1.4M 0.0k nvme5n7 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 117.6k 140.0k 0.0k 1.1M 1.4M 0.0k nvme5n8 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 152.8k 132.8k 0.0k 1.5M 1.3M 0.0k nvme5n9 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 114.4k 115.2k 0.0k 1.1M 1.1M 0.0k nvme8n1 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 163.2k 159.2k 0.0k 1.6M 1.6M 0.0k nvme8n10 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 136.8k 148.8k 0.0k 1.3M 1.5M 0.0k nvme8n11 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 187.6k 148.8k 0.0k 1.8M 1.5M 0.0k nvme8n12 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 95.2k 83.2k 0.0k 952.0k 832.0k 0.0k nvme8n13 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 133.6k 141.6k 0.0k 1.3M 1.4M 0.0k nvme8n14 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 84.8k 56.0k 0.0k 848.0k 560.0k 0.0k nvme8n15 2024-05-21T12:39:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 9.00 158.8k 141.6k 0.0k 1.6M 1.4M 0.0k nvme8n16 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 144.8k 115.2k 0.0k 1.4M 1.1M 0.0k nvme8n17 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 127.6k 144.4k 0.0k 1.2M 1.4M 0.0k nvme8n18 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 156.0k 138.0k 0.0k 1.5M 1.3M 0.0k nvme8n19 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 116.4k 158.4k 0.0k 1.1M 1.5M 0.0k nvme8n2 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 148.0k 152.4k 0.0k 1.4M 1.5M 0.0k nvme8n20 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 64.8k 88.0k 0.0k 648.0k 880.0k 0.0k nvme8n3 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 139.2k 116.4k 0.0k 1.4M 1.1M 0.0k nvme8n4 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 112.4k 132.4k 0.0k 1.1M 1.3M 0.0k nvme8n5 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 153.2k 140.4k 0.0k 1.5M 1.4M 0.0k nvme8n6 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 104.0k 172.8k 0.0k 1.0M 1.7M 0.0k nvme8n7 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 134.4k 176.0k 0.0k 1.3M 1.7M 0.0k nvme8n8 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 9.10 98.4k 69.6k 0.0k 984.0k 696.0k 0.0k nvme8n9 2024-05-21T12:39:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:45.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:45 smithi092 ceph-mon[33665]: pgmap v806: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 506 op/s 2024-05-21T12:39:45.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:45 smithi045 ceph-mon[30165]: pgmap v806: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 506 op/s 2024-05-21T12:39:45.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:45 smithi072 ceph-mon[33912]: pgmap v806: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 506 op/s 2024-05-21T12:39:48.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:47 smithi092 ceph-mon[33665]: pgmap v807: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 593 op/s 2024-05-21T12:39:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:47 smithi045 ceph-mon[30165]: pgmap v807: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 593 op/s 2024-05-21T12:39:48.326 INFO:tasks.nvmeof.[nvmeof.thrasher]:done waiting before reviving 2024-05-21T12:39:48.326 INFO:tasks.nvmeof.[nvmeof.thrasher]:display and verify stats: 2024-05-21T12:39:48.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:39:48.360 INFO:teuthology.orchestra.run.smithi045.stdout:× ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service - Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:39:48.360 INFO:teuthology.orchestra.run.smithi045.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:39:48.360 INFO:teuthology.orchestra.run.smithi045.stdout: Active: failed (Result: exit-code) since Tue 2024-05-21 12:37:33 UTC; 2min 14s ago 2024-05-21T12:39:48.360 INFO:teuthology.orchestra.run.smithi045.stdout: Duration: 1min 10.301s 2024-05-21T12:39:48.360 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 75063 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.360 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 75064 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76407 ExecStop=/bin/bash -c bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.stop (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76489 ExecStopPost=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.poststop (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76521 ExecStopPost=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout: Main PID: 75126 (code=exited, status=137) 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout: CPU: 1min 23.403s 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:22 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[75126]: [21-May-2024 12:37:22] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log.gz 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:32 smithi045 bash[76434]: time="2024-05-21T12:37:32Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a in 10 seconds, resorting to SIGKILL" 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:32 smithi045 podman[76434]: 2024-05-21 12:37:32.879310919 +0000 UTC m=+10.164228040 container died f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, distribution-scope=public, version=1.2.9, io.spdk.git.branch=undefined, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.git.branch=tags/1.2.9, io.ceph.version=1.2.9, io.ceph.component=ceph-nvmeof, io.ceph.maintainer=Ceph Developers , vcs-type=git, build-date=2024-05-19 17:57:19 UTC, name=ceph-nvmeof, io.spdk.build.ceph-release=19.0.0, release=, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.build.target-arch=, io.spdk.version=24.01, vendor=Red Hat, Inc., io.ceph.nvmeof=None, io.buildah.version=1.29.0, summary=Ceph NVMe over Fabrics Gateway, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, org.centos.version=9.0-21.el9, io.spdk.build.makeflags=, architecture=x86_64, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.repo=https://github.com/ceph/spdk.git, url=https://github.com/ceph/ceph-nvmeof, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, com.redhat.component=ubi9-container, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers ) 2024-05-21T12:39:48.361 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:33 smithi045 podman[76434]: 2024-05-21 12:37:33.496074996 +0000 UTC m=+10.780992108 container cleanup f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.spdk.build.makeflags=, io.spdk.build.target-arch=, io.spdk.git.branch=undefined, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.buildah.version=1.29.0, io.ceph.version=1.2.9, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.build.pkgdep-args=--rbd, summary=Ceph NVMe over Fabrics Gateway, build-date=2024-05-19 17:57:19 UTC, version=1.2.9, architecture=x86_64, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, name=ceph-nvmeof, org.centos.version=9.0-21.el9, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, io.ceph.component=ceph-nvmeof, distribution-scope=public, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, release=, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.maintainer=Ceph Developers , io.spdk.build.ceph-release=19.0.0, vcs-type=git, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.git.branch=tags/1.2.9, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.nvmeof=None, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi9-container, maintainer=Ceph Developers , io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.version=24.01) 2024-05-21T12:39:48.362 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:33 smithi045 bash[76434]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a 2024-05-21T12:39:48.362 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:33 smithi045 podman[76477]: 2024-05-21 12:37:33.747621632 +0000 UTC m=+0.866904792 container remove f187d3c6c75020f388cc7adba3812621ec6b687a26ad8d3c7e866ff9fd3ce800 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vendor=Red Hat, Inc., build-date=2024-05-19 17:57:19 UTC, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, url=https://github.com/ceph/ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.component=ceph-nvmeof, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.version=24.01, release=, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.target-arch=, io.buildah.version=1.29.0, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vcs-type=git, io.ceph.version=1.2.9, name=ceph-nvmeof, distribution-scope=public, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.repo=https://github.com/ceph/spdk.git, com.redhat.component=ubi9-container, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.maintainer=Ceph Developers , io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.git.branch=tags/1.2.9, io.spdk.build.ceph-release=19.0.0, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.nvmeof=None, maintainer=Ceph Developers , io.spdk.build.pkgdep-args=--rbd, version=1.2.9, architecture=x86_64, org.centos.version=9.0-21.el9, summary=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.git.branch=undefined) 2024-05-21T12:39:48.362 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:33 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:39:48.362 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:33 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Failed with result 'exit-code'. 2024-05-21T12:39:48.362 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:33 smithi045 systemd[1]: Stopped Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:39:48.362 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:37:33 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Consumed 1min 23.403s CPU time. 2024-05-21T12:39:48.366 DEBUG:teuthology.orchestra.run:got remote process result: 3 2024-05-21T12:39:48.367 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:39:48.388 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:47 smithi072 ceph-mon[33912]: pgmap v807: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 593 op/s 2024-05-21T12:39:48.398 INFO:teuthology.orchestra.run.smithi072.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service - Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:39:48.398 INFO:teuthology.orchestra.run.smithi072.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:39:48.398 INFO:teuthology.orchestra.run.smithi072.stdout: Active: active (running) since Tue 2024-05-21 12:36:12 UTC; 3min 35s ago 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56908 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56909 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: Main PID: 56971 (conmon) 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: Tasks: 126 (limit: 202814) 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: Memory: 152.7M 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: CPU: 3min 42.772s 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: ├─libpod-payload-8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56973 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56975 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56993 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.b --gateway-address 172.21.15.72:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.72:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.b -k /etc/ceph/keyring 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─57044 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:39:48.399 INFO:teuthology.orchestra.run.smithi072.stdout: │ └─57834 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:39:48.400 INFO:teuthology.orchestra.run.smithi072.stdout: └─runtime 2024-05-21T12:39:48.400 INFO:teuthology.orchestra.run.smithi072.stdout: └─56971 /usr/bin/conmon --api-version 1 -c 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -u 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata -p /run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:39:48.400 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:39:48.400 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:39:48.400 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:39:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:39:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:39:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:39:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:39:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: OPTIMIZED 2024-05-21T12:39:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:39:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:49880 2024-05-21T12:39:48.401 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:37:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [2024-05-21 12:37:54.065336] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000217 seconds 2024-05-21T12:39:48.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout:× ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service - Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Active: failed (Result: exit-code) since Tue 2024-05-21 12:37:45 UTC; 2min 3s ago 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Duration: 13min 37.144s 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53502 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 53503 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 57649 ExecStop=/bin/bash -c bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.stop (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 57719 ExecStopPost=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.poststop (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 57749 ExecStopPost=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:39:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: Main PID: 53679 (code=exited, status=137) 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: CPU: 13min 39.006s 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[53679]: [21-May-2024 12:37:34] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log.gz 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:44 smithi092 bash[57676]: time="2024-05-21T12:37:44Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c in 10 seconds, resorting to SIGKILL" 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:44 smithi092 podman[57676]: 2024-05-21 12:37:44.403094466 +0000 UTC m=+10.186405340 container died 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.ceph.component=ceph-nvmeof, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, url=https://github.com/ceph/ceph-nvmeof, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, architecture=x86_64, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, org.centos.version=9.0-21.el9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, vcs-type=git, name=ceph-nvmeof, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vendor=Red Hat, Inc., com.redhat.component=ubi9-container, release=, io.openshift.wants=ceph,rbd, distribution-scope=public, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.git.repo=https://github.com/ceph/spdk.git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.ceph-release=19.0.0, io.ceph.maintainer=Ceph Developers , maintainer=Ceph Developers , version=1.2.9, build-date=2024-05-19 17:57:19 UTC, io.ceph.nvmeof=None, io.buildah.version=1.29.0, io.spdk.git.branch=undefined, io.spdk.build.pkgdep-args=--rbd, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.version=1.2.9, io.spdk.build.target-arch=, summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, io.spdk.version=24.01, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.ceph.git.branch=tags/1.2.9) 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:44 smithi092 podman[57676]: 2024-05-21 12:37:44.857670501 +0000 UTC m=+10.640981385 container cleanup 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.spdk.build.makeflags=, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., release=, io.ceph.maintainer=Ceph Developers , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.git.branch=tags/1.2.9, architecture=x86_64, io.ceph.component=ceph-nvmeof, vendor=Red Hat, Inc., io.ceph.version=1.2.9, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., version=1.2.9, build-date=2024-05-19 17:57:19 UTC, com.redhat.component=ubi9-container, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, summary=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vcs-type=git, io.openshift.wants=ceph,rbd, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.build.target-arch=, io.spdk.git.branch=undefined, io.spdk.version=24.01, name=ceph-nvmeof, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.nvmeof=None, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.ceph-release=19.0.0, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.buildah.version=1.29.0, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, distribution-scope=public, org.centos.version=9.0-21.el9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.pkgdep-args=--rbd, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.url=https://github.com/ceph/ceph-nvmeof, maintainer=Ceph Developers , url=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:44 smithi092 bash[57676]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:45 smithi092 podman[57703]: 2024-05-21 12:37:45.15111715 +0000 UTC m=+0.747780033 container remove 5e5a8ffd2ba657ffe41eda2bb5a15b2759d44f04f6fbcfe926782f77ac65b14d (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, name=ceph-nvmeof, io.openshift.wants=ceph,rbd, io.spdk.build.makeflags=, io.buildah.version=1.29.0, io.spdk.build.ceph-release=19.0.0, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.target-arch=, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.ceph.url=https://github.com/ceph/ceph-nvmeof, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, release=, io.ceph.nvmeof=None, vcs-type=git, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, org.centos.version=9.0-21.el9, io.spdk.build.pkgdep-args=--rbd, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.branch=undefined, summary=Ceph NVMe over Fabrics Gateway, io.spdk.version=24.01, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, architecture=x86_64, io.ceph.version=1.2.9, url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, build-date=2024-05-19 17:57:19 UTC, io.ceph.component=ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.maintainer=Ceph Developers , version=1.2.9, com.redhat.component=ubi9-container, io.spdk.git.repo=https://github.com/ceph/spdk.git, distribution-scope=public) 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:45 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:45 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Failed with result 'exit-code'. 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:45 smithi092 systemd[1]: Stopped Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:39:48.436 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:37:45 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Consumed 13min 39.006s CPU time. 2024-05-21T12:39:48.437 DEBUG:teuthology.orchestra.run:got remote process result: 3 2024-05-21T12:39:48.438 DEBUG:teuthology.orchestra.run.smithi137:> ceph orch ls && ceph orch ps --daemon-type nvmeof && ceph health detail && ceph -s && sudo nvme list-subsys /dev/nvme5n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n20 | grep 'live optimized' 2024-05-21T12:39:48.751 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T12:39:48.752 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 1/1 4m ago 21m smithi045=x;count:1 2024-05-21T12:39:48.752 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 4m ago 21m smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:39:48.752 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.mypool ?:4420,5500,8009 1/3 4m ago 16m smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:39:48.752 INFO:teuthology.orchestra.run.smithi137.stdout:osd 8 4m ago - 2024-05-21T12:39:49.055 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:39:49.055 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.a smithi045 *:5500,4420,8009 error 4m ago 15m - - 2024-05-21T12:39:49.055 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.b smithi072 *:5500,4420,8009 error 4m ago 15m - - 2024-05-21T12:39:49.055 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.c smithi092 *:5500,4420,8009 running (15m) 4m ago 15m 160M - 1.2.9 30dd525ace30 5e5a8ffd2ba6 2024-05-21T12:39:49.410 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:39:49.410 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:39:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:39:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:39:49.765 INFO:teuthology.orchestra.run.smithi137.stdout: cluster: 2024-05-21T12:39:49.765 INFO:teuthology.orchestra.run.smithi137.stdout: id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:39:49.765 INFO:teuthology.orchestra.run.smithi137.stdout: health: HEALTH_WARN 2024-05-21T12:39:49.765 INFO:teuthology.orchestra.run.smithi137.stdout: 2 failed cephadm daemon(s) 2024-05-21T12:39:49.765 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:39:49.765 INFO:teuthology.orchestra.run.smithi137.stdout: services: 2024-05-21T12:39:49.766 INFO:teuthology.orchestra.run.smithi137.stdout: mon: 3 daemons, quorum a,c,b (age 21m) 2024-05-21T12:39:49.766 INFO:teuthology.orchestra.run.smithi137.stdout: mgr: x(active, since 22m) 2024-05-21T12:39:49.766 INFO:teuthology.orchestra.run.smithi137.stdout: osd: 8 osds: 8 up (since 17m), 8 in (since 18m) 2024-05-21T12:39:49.766 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:39:49.766 INFO:teuthology.orchestra.run.smithi137.stdout: data: 2024-05-21T12:39:49.766 INFO:teuthology.orchestra.run.smithi137.stdout: pools: 2 pools, 33 pgs 2024-05-21T12:39:49.766 INFO:teuthology.orchestra.run.smithi137.stdout: objects: 485 objects, 1.0 GiB 2024-05-21T12:39:49.766 INFO:teuthology.orchestra.run.smithi137.stdout: usage: 4.0 GiB used, 711 GiB / 715 GiB avail 2024-05-21T12:39:49.767 INFO:teuthology.orchestra.run.smithi137.stdout: pgs: 33 active+clean 2024-05-21T12:39:49.767 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:39:49.767 INFO:teuthology.orchestra.run.smithi137.stdout: io: 2024-05-21T12:39:49.767 INFO:teuthology.orchestra.run.smithi137.stdout: client: 7.2 MiB/s rd, 7.2 MiB/s wr, 257 op/s rd, 256 op/s wr 2024-05-21T12:39:49.767 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:39:49.795 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.814 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.832 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.850 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.869 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.887 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.905 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.923 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:49.995 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.013 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.031 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.049 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.067 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.085 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.104 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.122 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.140 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.159 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.177 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.195 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.214 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.232 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.251 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:49 smithi092 ceph-mon[33665]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:39:50.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:49 smithi092 ceph-mon[33665]: pgmap v808: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 514 op/s 2024-05-21T12:39:50.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:49 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/3653455402' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:39:50.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:49 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/2497708874' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:39:50.269 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.287 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.306 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:49 smithi045 ceph-mon[30165]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:39:50.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:49 smithi045 ceph-mon[30165]: pgmap v808: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 514 op/s 2024-05-21T12:39:50.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:49 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/3653455402' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:39:50.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:49 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/2497708874' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:39:50.324 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.342 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.361 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.379 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.398 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.419 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:49 smithi072 ceph-mon[33912]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:39:50.419 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:49 smithi072 ceph-mon[33912]: pgmap v808: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 514 op/s 2024-05-21T12:39:50.420 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:49 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/3653455402' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:39:50.420 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:49 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/2497708874' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:39:50.420 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.434 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.453 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.471 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.489 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.507 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.526 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.544 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.562 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.581 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.599 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.617 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.636 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.654 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.672 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.690 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.708 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.726 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.745 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.763 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.782 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.799 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.818 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.944 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.944 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.945 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:39:50.945 INFO:tasks.nvmeof.[nvmeof.thrasher]:reviving nvmeof.a 2024-05-21T12:39:50.945 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.a:Restarting nvmeof.nvmeof.a (starting--it wasn't running)... 2024-05-21T12:39:50.945 DEBUG:teuthology.orchestra.run.smithi045:nvmeof.nvmeof.a> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:39:50.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl start ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:39:51.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:50 smithi092 ceph-mon[33665]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:39:51.261 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:50 smithi045 ceph-mon[30165]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:39:51.261 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 systemd[1]: Starting Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:39:51.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:50 smithi072 ceph-mon[33912]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:39:51.534 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 podman[77002]: 2024-05-21 12:39:51.164900653 +0000 UTC m=+0.019580655 image pull 30dd525ace30bc703cdf4a290b3be2b24f6ca7b74757424a10799eaf370636b4 quay.io/ceph/nvmeof:1.2 2024-05-21T12:39:51.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 podman[77002]: 2024-05-21 12:39:51.281877276 +0000 UTC m=+0.136557258 container create 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, name=ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.target-arch=, vcs-type=git, distribution-scope=public, io.buildah.version=1.29.0, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.nvmeof=None, release=, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.version=1.2.9, summary=Ceph NVMe over Fabrics Gateway, vendor=Red Hat, Inc., io.ceph.component=ceph-nvmeof, io.spdk.build.ceph-release=19.0.0, maintainer=Ceph Developers , io.openshift.wants=ceph,rbd, url=https://github.com/ceph/ceph-nvmeof, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.maintainer=Ceph Developers , build-date=2024-05-19 17:57:19 UTC, io.ceph.git.branch=tags/1.2.9, io.spdk.version=24.01, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.pkgdep-args=--rbd, version=1.2.9, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, com.redhat.component=ubi9-container, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, org.centos.version=9.0-21.el9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.spdk.git.branch=undefined, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, architecture=x86_64) 2024-05-21T12:39:51.535 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 podman[77002]: 2024-05-21 12:39:51.53473317 +0000 UTC m=+0.389413165 container init 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.spdk.build.makeflags=, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.spdk.git.branch=undefined, io.ceph.summary=Ceph NVMe over Fabrics Gateway, maintainer=Ceph Developers , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.pkgdep-args=--rbd, name=ceph-nvmeof, version=1.2.9, io.ceph.component=ceph-nvmeof, io.spdk.build.target-arch=, url=https://github.com/ceph/ceph-nvmeof, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.wants=ceph,rbd, io.ceph.maintainer=Ceph Developers , build-date=2024-05-19 17:57:19 UTC, com.redhat.component=ubi9-container, io.buildah.version=1.29.0, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, org.centos.version=9.0-21.el9, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.ceph.nvmeof=None, io.spdk.version=24.01, io.ceph.git.branch=tags/1.2.9, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-type=git, release=, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, architecture=x86_64, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., summary=Ceph NVMe over Fabrics Gateway, vendor=Red Hat, Inc., distribution-scope=public, io.ceph.version=1.2.9, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.build.ceph-release=19.0.0) 2024-05-21T12:39:51.641 INFO:tasks.nvmeof.[nvmeof.thrasher]:reviving nvmeof.c 2024-05-21T12:39:51.641 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.c:Restarting nvmeof.nvmeof.c (starting--it wasn't running)... 2024-05-21T12:39:51.641 DEBUG:teuthology.orchestra.run.smithi092:nvmeof.nvmeof.c> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:39:51.643 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl start ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:39:51.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 podman[77002]: 2024-05-21 12:39:51.538594445 +0000 UTC m=+0.393274428 container start 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, version=1.2.9, io.ceph.component=ceph-nvmeof, url=https://github.com/ceph/ceph-nvmeof, summary=Ceph NVMe over Fabrics Gateway, io.openshift.wants=ceph,rbd, io.ceph.summary=Ceph NVMe over Fabrics Gateway, vendor=Red Hat, Inc., maintainer=Ceph Developers , vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.git.branch=tags/1.2.9, io.buildah.version=1.29.0, io.ceph.maintainer=Ceph Developers , org.centos.version=9.0-21.el9, io.spdk.build.ceph-release=19.0.0, io.ceph.nvmeof=None, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, release=, build-date=2024-05-19 17:57:19 UTC, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-type=git, name=ceph-nvmeof, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.target-arch=, io.spdk.version=24.01, io.ceph.version=1.2.9, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.git.branch=undefined, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, com.redhat.component=ubi9-container, architecture=x86_64, io.spdk.build.makeflags=, io.spdk.build.pkgdep-args=--rbd) 2024-05-21T12:39:51.811 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 bash[77002]: 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f 2024-05-21T12:39:51.812 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 systemd[1]: Started Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:39:51.941 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:51 smithi092 systemd[1]: Starting Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:39:52.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-mon[30165]: pgmap v809: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 526 op/s 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO utils.py:258: Initialize gateway log level to "INFO" 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO utils.py:271: Log files will be saved in /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a, using rotation 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:78: Using NVMeoF gateway version 1.2.9 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:81: Using SPDK version 24.01 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:84: Using vstart cluster version based on 19.0.0 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:87: NVMeoF gateway built on: 2024-05-19 17:57:19 UTC 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:90: NVMeoF gateway Git repository: https://github.com/ceph/ceph-nvmeof 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:93: NVMeoF gateway Git branch: tags/1.2.9 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:96: NVMeoF gateway Git commit: 06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:102: SPDK Git repository: https://github.com/ceph/spdk.git 2024-05-21T12:39:52.064 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:105: SPDK Git branch: undefined 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:108: SPDK Git commit: dade5b4c96a647e63ef96d9a5e05f3b64ea0598a 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:59: Using configuration file /src/ceph-nvmeof.conf 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:61: ====================================== Configuration file content ====================================== 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: # This file is generated by cephadm. 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: [gateway] 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: name = client.nvmeof.nvmeof.a 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: group = 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: addr = 172.21.15.45 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: port = 5500 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: enable_auth = False 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: state_update_notify = True 2024-05-21T12:39:52.065 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: state_update_interval_sec = 5 2024-05-21T12:39:52.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: enable_spdk_discovery_controller = False 2024-05-21T12:39:52.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: enable_prometheus_exporter = True 2024-05-21T12:39:52.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: prometheus_exporter_ssl = False 2024-05-21T12:39:52.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: prometheus_port = 10008 2024-05-21T12:39:52.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: verify_nqns = True 2024-05-21T12:39:52.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: omap_file_lock_duration = 20 2024-05-21T12:39:52.066 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: omap_file_lock_retries = 30 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: omap_file_lock_retry_sleep_interval = 1.0 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: omap_file_update_reloads = 10 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: allowed_consecutive_spdk_ping_failures = 1 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: spdk_ping_interval_in_seconds = 2.0 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: ping_spdk_under_lock = False 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: enable_monitor_client = True 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: [gateway-logs] 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: log_level = INFO 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: log_files_enabled = True 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: log_files_rotation_enabled = True 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: verbose_log_messages = True 2024-05-21T12:39:52.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: max_log_file_size_in_mb = 10 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: max_log_files_count = 20 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: max_log_directory_backups = 10 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: log_directory = /var/log/ceph/ 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: [discovery] 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: addr = 172.21.15.45 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: port = 8009 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: [ceph] 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: pool = mypool 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: config_file = /etc/ceph/ceph.conf 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: id = nvmeof.nvmeof.a 2024-05-21T12:39:52.068 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: [mtls] 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: server_key = ./server.key 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: client_key = ./client.key 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: server_cert = ./server.crt 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: client_cert = ./client.crt 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: [spdk] 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: tgt_path = /usr/local/bin/nvmf_tgt 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: rpc_socket_dir = /var/tmp/ 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: rpc_socket_name = spdk.sock 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: timeout = 60.0 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: bdevs_per_cluster = 32 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: log_level = WARNING 2024-05-21T12:39:52.069 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: conn_retries = 10 2024-05-21T12:39:52.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: transports = tcp 2024-05-21T12:39:52.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: transport_tcp_options = {"in_capsule_data_size": 8192, "max_io_qpairs_per_ctrlr": 7} 2024-05-21T12:39:52.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: 2024-05-21T12:39:52.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: [monitor] 2024-05-21T12:39:52.070 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:65: timeout = 1.0 2024-05-21T12:39:52.073 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO config.py:66: ======================================================================================================== 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:90: Starting gateway client.nvmeof.nvmeof.a 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:157: Starting serve, monitor client version: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:236: Starting /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.a --gateway-address 172.21.15.45:5500 --gateway-pool mypool --gateway-group --monitor-group-address 172.21.15.45:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.a -k /etc/ceph/keyring 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:240: monitor client process id: 19 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:146: MonitorGroup server is listening on 172.21.15.45:5499 for group id 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.891+0000 7f7d7b3415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.891+0000 7f7d7b3415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() gateway name: client.nvmeof.nvmeof.a pool:mypool group: address: 172.21.15.45:5500 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.891+0000 7f7d7b3415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() nvmeof Registered monc callback 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.893+0000 7f7d7b3415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() monc.authentication done 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.894+0000 7f7d7b3415c0 0 nvmeofgw void NVMeofGwMonitorClient::tick() 2024-05-21T12:39:52.074 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.895+0000 7f7d7b3415c0 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.895+0000 7f7d7b3415c0 0 nvmeofgw void NVMeofGwMonitorClient::send_beacon() sending beacon as gid 15423 availability 0 osdmap_epoch 0 gwmap_epoch 0 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.895+0000 7f7d7b3415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() Complete. 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.895+0000 7f7d772de640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.895+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 135 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.895+0000 7f7d772de640 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.895+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_group_id: 0 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:135: Gateway client.nvmeof.nvmeof.a group id=0 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:149: Stopping the MonitorGroup server... 2024-05-21T12:39:52.075 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.076 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.077 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.078 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.896+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.079 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.897+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.899+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.900+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:152: The MonitorGroup gRPC server stopped... 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:349: SPDK Target Path: /usr/local/bin/nvmf_tgt 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:351: SPDK Socket: /var/tmp/spdk.sock 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:357: Starting /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.902+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:370: SPDK process id: 70 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.903+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO server.py:371: Attempting to initialize SPDK: rpc_socket: /var/tmp/spdk.sock, conn_retries: 300, timeout: 60.0 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: INFO: Setting log level to WARNING 2024-05-21T12:39:52.080 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:51] INFO client.py:110: Setting log level to WARNING 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.905+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.906+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.908+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.909+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.911+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.912+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.914+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.916+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:51.918676] Starting SPDK v24.01 / DPDK 23.11.0 initialization... 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:51.918756] [ DPDK EAL parameters: nvmf --no-shconf -c 0x1 --no-pci --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --iova-mode=pa --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid70 ] 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.917+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.919+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.081 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.921+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.922+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.924+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.925+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.927+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.928+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.930+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.931+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.933+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.935+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.936+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.082 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.938+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.939+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.941+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.943+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.944+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.946+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.947+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.949+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.951+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.952+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.083 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.954+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.955+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.957+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.958+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.960+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.961+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.963+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.964+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.966+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.967+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.969+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.970+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.972+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.084 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.973+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.975+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.976+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.978+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.979+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.981+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.982+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.984+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.985+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.987+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.988+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.990+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.991+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.085 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.993+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.086 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.994+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.086 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.996+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.086 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:51 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.997+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.086 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:51.999+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.086 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.000+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.086 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.002+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.003+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.005+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.006+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.008+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.009+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.011+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.012+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.014+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.015+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.017+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.018+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.087 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.020+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.021+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.023+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.025+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.026+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.028+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.029+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.031+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.033982] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.032+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.035+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.037+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.039+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.040+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.088 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.042+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.044+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.046+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.048+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.049+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.051+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.053+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.054+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.056+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.058+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.060+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.062+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.064+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.089 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.065+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.090 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.067+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.090 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.068+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.090 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.070+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.090 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.072+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.090 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.073+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.224 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:51 smithi092 ceph-mon[33665]: pgmap v809: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 526 op/s 2024-05-21T12:39:52.224 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:51 smithi092 podman[58239]: 2024-05-21 12:39:51.848549372 +0000 UTC m=+0.018819020 image pull 30dd525ace30bc703cdf4a290b3be2b24f6ca7b74757424a10799eaf370636b4 quay.io/ceph/nvmeof:1.2 2024-05-21T12:39:52.224 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:51 smithi092 podman[58239]: 2024-05-21 12:39:51.979451074 +0000 UTC m=+0.149720707 container create 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.ceph.nvmeof=None, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.version=24.01, com.redhat.component=ubi9-container, io.openshift.wants=ceph,rbd, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.ceph-release=19.0.0, io.ceph.component=ceph-nvmeof, io.spdk.build.makeflags=, org.centos.version=9.0-21.el9, release=, io.ceph.git.branch=tags/1.2.9, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, maintainer=Ceph Developers , name=ceph-nvmeof, version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, distribution-scope=public, io.spdk.build.pkgdep-args=--rbd, vendor=Red Hat, Inc., io.ceph.maintainer=Ceph Developers , vcs-type=git, summary=Ceph NVMe over Fabrics Gateway, io.ceph.version=1.2.9, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.branch=undefined, build-date=2024-05-19 17:57:19 UTC, architecture=x86_64, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., url=https://github.com/ceph/ceph-nvmeof, io.spdk.git.repo=https://github.com/ceph/spdk.git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.build.target-arch=, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.buildah.version=1.29.0, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.url=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:39:52.315 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.075+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.315 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.077+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.078+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.080+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.082+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.084+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.085+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.087+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.089+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.090593] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 2024-05-21T12:39:52.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.091+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.093+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.095+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.097+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.099+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.101+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.103+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.104+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.106+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.107+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.109+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.110+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.112+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.113+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.115+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.117+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.119+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.121+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.123+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.125+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.128+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.129+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.131+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.319 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:51 smithi072 ceph-mon[33912]: pgmap v809: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 526 op/s 2024-05-21T12:39:52.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.133+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.134+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.136+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.138+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.321 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.139+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.141+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.142+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.144+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.145+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.147+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.148+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.150+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.151+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.153+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.322 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.155+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.156+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.157+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.159+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.161+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.162+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.164+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.165+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.167+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.168+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.170+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.171+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.323 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.173+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.174+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.176+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.177+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.179+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.180+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.182+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.183+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.185+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.186+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.188+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.189+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.324 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.191+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.192+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.194+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.195+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.197+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.198+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.200+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.201+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.203+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.204+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.206+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.207+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.209+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.210+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.325 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.212+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.213+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.215+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.216+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.218+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.219+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.221+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.222+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.224+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.226+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.227+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.229+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.326 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.230+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.232+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.233+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.235+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.236+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.238+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.239+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.241+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.242+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.244+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.245+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.247+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.248+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.327 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.250+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.251+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.253+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.255+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.256+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.258+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.259+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.260+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.262+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.263+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.265+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.266+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.328 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.268+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.269+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.271+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.272+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.274+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.275+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.277+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.279+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.280+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.282+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.329 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.283+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.285+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.286+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.288+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.289+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.291+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.292+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.294+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.295+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.297+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.298+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.330 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.300+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.301+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.303+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.304+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.306671] tcp.c: 659:nvmf_tcp_create: *NOTICE*: *** TCP Transport Init *** 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.306+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.307+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.309+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.310+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.312+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.314+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.315+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.331 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.317+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.338 INFO:tasks.nvmeof.[nvmeof.thrasher]:waiting for 89 secs before thrashing 2024-05-21T12:39:52.497 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 podman[58239]: 2024-05-21 12:39:52.223339938 +0000 UTC m=+0.393609572 container init 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.summary=Ceph NVMe over Fabrics Gateway, summary=Ceph NVMe over Fabrics Gateway, io.ceph.maintainer=Ceph Developers , org.centos.version=9.0-21.el9, io.ceph.url=https://github.com/ceph/ceph-nvmeof, vcs-type=git, io.spdk.version=24.01, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.git.branch=undefined, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.ceph-release=19.0.0, com.redhat.component=ubi9-container, version=1.2.9, io.ceph.nvmeof=None, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.makeflags=, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, name=ceph-nvmeof, io.ceph.version=1.2.9, build-date=2024-05-19 17:57:19 UTC, io.ceph.component=ceph-nvmeof, release=, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.openshift.wants=ceph,rbd, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.pkgdep-args=--rbd, io.ceph.git.branch=tags/1.2.9, architecture=x86_64, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, maintainer=Ceph Developers , io.spdk.build.target-arch=, url=https://github.com/ceph/ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, vendor=Red Hat, Inc., io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.buildah.version=1.29.0, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a) 2024-05-21T12:39:52.498 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 podman[58239]: 2024-05-21 12:39:52.226900393 +0000 UTC m=+0.397170027 container start 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.build.pkgdep-args=--rbd, io.openshift.wants=ceph,rbd, io.ceph.summary=Ceph NVMe over Fabrics Gateway, org.centos.version=9.0-21.el9, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.version=24.01, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., url=https://github.com/ceph/ceph-nvmeof, release=, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, version=1.2.9, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.nvmeof=None, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.29.0, io.ceph.git.branch=tags/1.2.9, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, maintainer=Ceph Developers , architecture=x86_64, io.ceph.component=ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, build-date=2024-05-19 17:57:19 UTC, io.spdk.build.ceph-release=19.0.0, io.spdk.build.makeflags=, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, com.redhat.component=ubi9-container, distribution-scope=public, io.ceph.maintainer=Ceph Developers , io.ceph.version=1.2.9, name=ceph-nvmeof, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vendor=Red Hat, Inc., io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.git.branch=undefined, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, summary=Ceph NVMe over Fabrics Gateway, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec) 2024-05-21T12:39:52.498 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 bash[58239]: 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 2024-05-21T12:39:52.498 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 systemd[1]: Started Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:39:52.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO server.py:268: Discovery service process id: 858 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.319+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO server.py:264: Starting ceph nvmeof discovery service 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.320+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.322+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.324+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.326+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.328+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.329+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.331+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.333+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO cephutils.py:74: Connected to Ceph with version "19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)" 2024-05-21T12:39:52.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:86: Requested huge pages count is 2048 2024-05-21T12:39:52.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:105: Actual huge pages count is 4096 2024-05-21T12:39:52.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:190: NVMeoF bdevs per cluster: 32 2024-05-21T12:39:52.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.335+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.336+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO discovery.py:322: log pages info from omap: nvmeof.state 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO server.py:195: Prometheus endpoint is enabled 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO prometheus.py:98: Prometheus exporter running in http mode, listening on port 10008 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 1 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.569 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 2 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 4 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.570 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 5 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 6 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 7 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.571 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 8 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 9 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 10 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.572 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 11 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 12 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 13 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.573 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 14 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 15 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 16 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.574 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:39:52.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 1 2024-05-21T12:39:52.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.575 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 2 2024-05-21T12:39:52.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:39:52.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.576 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 4 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 5 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.577 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 6 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 7 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 8 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.578 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 9 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 10 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 11 2024-05-21T12:39:52.579 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 12 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 13 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 14 2024-05-21T12:39:52.580 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 15 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 16 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.581 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 1 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 2 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.582 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 4 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 5 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 6 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 7 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.583 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 8 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 9 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 10 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.584 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 11 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 12 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 13 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.585 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 14 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 15 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 16 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: ], , client address: IPv4 172.21.15.45:34848 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode1 not found 2024-05-21T12:39:52.586 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode2 not found 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:2648: Received request to get gateway's info 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO prometheus.py:131: Stats for all bdevs will be provided 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.350+0000 7f7d772de640 0 ms_deliver_dispatch: unhandled message 0x560dcdfbd180 nvmeofgwmap magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.350+0000 7f7d772de640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 4 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.350+0000 7f7d772de640 0 ms_deliver_dispatch: unhandled message 0x560dcdfadd40 mon_map magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.351+0000 7f7d772de640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 41 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.351+0000 7f7d772de640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.351+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 136 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:39:52.587 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.351+0000 7f7d772de640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) got_old_gw_state: NvmeGwState { group id: 0 gw_map_epoch 135 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }got_new_gw_state: NvmeGwState { group id: 0 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: 2024-05-21T12:39:52.351+0000 7f7d772de640 0 ms_deliver_dispatch: unhandled message 0x560dcdfbd500 nvmeofgwmap magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO discovery.py:330: discovery addr: 172.21.15.45 port: 8009 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.395806] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.45:0/1418595029, rc 0 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.396493] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.45:0/1418595029 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.45:0/1418595029' anagrp=1 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.410861] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad 2024-05-21T12:39:52.588 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.431792] nvmf_rpc.c: 275:rpc_nvmf_get_subsystems: *WARNING*: rpc_nvmf_get_subsystems: deprecated feature listener.transport is deprecated in favor of trtype to be removed in v24.05 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.447187] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.45:0/2595843178, rc 0 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.447290] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.45:0/2595843178 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.45:0/2595843178' anagrp=2 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.461392] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 2024-05-21T12:39:52.589 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.494888] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.561242] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:39:52.590 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 2024-05-21T12:39:52.749 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO utils.py:258: Initialize gateway log level to "INFO" 2024-05-21T12:39:52.749 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO utils.py:271: Log files will be saved in /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c, using rotation 2024-05-21T12:39:52.749 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:78: Using NVMeoF gateway version 1.2.9 2024-05-21T12:39:52.749 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:81: Using SPDK version 24.01 2024-05-21T12:39:52.749 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:84: Using vstart cluster version based on 19.0.0 2024-05-21T12:39:52.749 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:87: NVMeoF gateway built on: 2024-05-19 17:57:19 UTC 2024-05-21T12:39:52.749 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:90: NVMeoF gateway Git repository: https://github.com/ceph/ceph-nvmeof 2024-05-21T12:39:52.749 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:93: NVMeoF gateway Git branch: tags/1.2.9 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:96: NVMeoF gateway Git commit: 06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:102: SPDK Git repository: https://github.com/ceph/spdk.git 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:105: SPDK Git branch: undefined 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:108: SPDK Git commit: dade5b4c96a647e63ef96d9a5e05f3b64ea0598a 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:59: Using configuration file /src/ceph-nvmeof.conf 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:61: ====================================== Configuration file content ====================================== 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: # This file is generated by cephadm. 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: [gateway] 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: name = client.nvmeof.nvmeof.c 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: group = 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: addr = 172.21.15.92 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: port = 5500 2024-05-21T12:39:52.750 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: enable_auth = False 2024-05-21T12:39:52.751 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: state_update_notify = True 2024-05-21T12:39:52.751 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: state_update_interval_sec = 5 2024-05-21T12:39:52.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: enable_spdk_discovery_controller = False 2024-05-21T12:39:52.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: enable_prometheus_exporter = True 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: prometheus_exporter_ssl = False 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: prometheus_port = 10008 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: verify_nqns = True 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: omap_file_lock_duration = 20 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: omap_file_lock_retries = 30 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: omap_file_lock_retry_sleep_interval = 1.0 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: omap_file_update_reloads = 10 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: allowed_consecutive_spdk_ping_failures = 1 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: spdk_ping_interval_in_seconds = 2.0 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: ping_spdk_under_lock = False 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: enable_monitor_client = True 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: 2024-05-21T12:39:52.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: [gateway-logs] 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: log_level = INFO 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: log_files_enabled = True 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: log_files_rotation_enabled = True 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: verbose_log_messages = True 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: max_log_file_size_in_mb = 10 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: max_log_files_count = 20 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: max_log_directory_backups = 10 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: log_directory = /var/log/ceph/ 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: [discovery] 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: addr = 172.21.15.92 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: port = 8009 2024-05-21T12:39:52.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: [ceph] 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: pool = mypool 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: config_file = /etc/ceph/ceph.conf 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: id = nvmeof.nvmeof.c 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: [mtls] 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: server_key = ./server.key 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: client_key = ./client.key 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: server_cert = ./server.crt 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: client_cert = ./client.crt 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: [spdk] 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: tgt_path = /usr/local/bin/nvmf_tgt 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: rpc_socket_dir = /var/tmp/ 2024-05-21T12:39:52.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: rpc_socket_name = spdk.sock 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: timeout = 60.0 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: bdevs_per_cluster = 32 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: log_level = WARNING 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: conn_retries = 10 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: transports = tcp 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: transport_tcp_options = {"in_capsule_data_size": 8192, "max_io_qpairs_per_ctrlr": 7} 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: [monitor] 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:65: timeout = 1.0 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO config.py:66: ======================================================================================================== 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:90: Starting gateway client.nvmeof.nvmeof.c 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:157: Starting serve, monitor client version: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:236: Starting /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.c --gateway-address 172.21.15.92:5500 --gateway-pool mypool --gateway-group --monitor-group-address 172.21.15.92:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.c -k /etc/ceph/keyring 2024-05-21T12:39:52.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:240: monitor client process id: 19 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:146: MonitorGroup server is listening on 172.21.15.92:5499 for group id 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.579+0000 7fea17eb95c0 0 nvmeofgw int NVMeofGwMonitorClient::init() 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.579+0000 7fea17eb95c0 0 nvmeofgw int NVMeofGwMonitorClient::init() gateway name: client.nvmeof.nvmeof.c pool:mypool group: address: 172.21.15.92:5500 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.579+0000 7fea17eb95c0 0 nvmeofgw int NVMeofGwMonitorClient::init() nvmeof Registered monc callback 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.582+0000 7fea17eb95c0 0 nvmeofgw int NVMeofGwMonitorClient::init() monc.authentication done 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.582+0000 7fea17eb95c0 0 nvmeofgw void NVMeofGwMonitorClient::tick() 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.583+0000 7fea17eb95c0 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.583+0000 7fea17eb95c0 0 nvmeofgw void NVMeofGwMonitorClient::send_beacon() sending beacon as gid 25217 availability 0 osdmap_epoch 0 gwmap_epoch 0 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.583+0000 7fea17eb95c0 0 nvmeofgw int NVMeofGwMonitorClient::init() Complete. 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.583+0000 7fea13e56640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.583+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:39:52.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 136 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.583+0000 7fea13e56640 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.583+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_group_id: 2 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:135: Gateway client.nvmeof.nvmeof.c group id=2 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:149: Stopping the MonitorGroup server... 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.759 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.760 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.761 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.762 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.763 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.763 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.763 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.764 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.764 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.584+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:52.764 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.585+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.586+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.588+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:152: The MonitorGroup gRPC server stopped... 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:349: SPDK Target Path: /usr/local/bin/nvmf_tgt 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:351: SPDK Socket: /var/tmp/spdk.sock 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:357: Starting /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.589+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.591+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:370: SPDK process id: 70 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO server.py:371: Attempting to initialize SPDK: rpc_socket: /var/tmp/spdk.sock, conn_retries: 300, timeout: 60.0 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: INFO: Setting log level to WARNING 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:52] INFO client.py:110: Setting log level to WARNING 2024-05-21T12:39:52.765 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.593+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.595+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.596+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.598+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.599+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.601+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.603+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.604+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.606+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:52.607936] Starting SPDK v24.01 / DPDK 23.11.0 initialization... 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:52.607992] [ DPDK EAL parameters: nvmf --no-shconf -c 0x1 --no-pci --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --iova-mode=pa --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid70 ] 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.608+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.766 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.609+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.610+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.612+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.614+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.616+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.617+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.619+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.620+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.622+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.623+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.625+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.626+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.628+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.629+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.767 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.631+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.632+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.634+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.635+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.637+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.638+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.640+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.641+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.643+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.644+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.646+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.647+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.649+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.651+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.768 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.652+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.654+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.655+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.657+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.658+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.660+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.661+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.662+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.664+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.665+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.667+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.669+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.670+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.671+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.673+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.674+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.676+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.677+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.679+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.680+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.682+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.683+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.685+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.686+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.688+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.689+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.691+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.692+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.694+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.695+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.697+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.699+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.700+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.702+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.703+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.705+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.706+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.708+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.709+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.711+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.712+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.714+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.715+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.717+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.718+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.720+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.721+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:52.723552] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.723+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.725+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.727+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.729+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.731+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.733+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.734+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.736+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.737+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.739+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.741+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.742+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.744+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.745+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.747+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.749+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.751+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.753+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.754+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:52.851 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:52.851 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.851 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:39:52.851 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.607517] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:39:52.851 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c 2024-05-21T12:39:52.851 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:52.851 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.851 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:39:52.852 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.632533] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:39:52.852 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 2024-05-21T12:39:52.852 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:52.852 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.852 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:39:52.852 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.653818] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:39:52.852 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.688363] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.45:0/1400773162, rc 0 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.688497] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.45:0/1400773162 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.45:0/1400773162' anagrp=3 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.703422] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.853 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.727919] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.750397] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.774997] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 2024-05-21T12:39:52.854 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.795993] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.818928] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:39:52.855 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.842298] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:39:52.856 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 2024-05-21T12:39:52.856 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:52.856 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:52.856 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:39:53.000 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/448261625' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:39:53.000 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.756+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.000 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.757+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.000 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.759+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.000 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.761+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.762+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.764+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.765+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.767+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.768+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.770+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.772+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.773+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.775+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.776+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:52.779505] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 2024-05-21T12:39:53.001 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.778+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.780+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.782+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.783+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.002 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.785+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.787+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.789+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.791+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.792+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.794+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.795+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.797+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.798+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.800+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.801+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.803+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.805+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.003 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.807+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.809+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.811+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.813+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.815+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.817+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.818+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.820+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.822+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.823+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.825+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.826+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.004 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.828+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.829+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.831+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.832+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.834+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.835+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.837+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.839+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.840+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.842+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.843+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.845+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.846+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.005 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.848+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.849+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.851+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.852+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.853+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.855+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.856+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.858+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.860+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.861+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.863+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.864+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.866+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.006 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.867+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.869+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.870+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.872+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.873+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.875+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.876+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.878+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.879+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.881+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.882+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.007 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.884+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.008 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.885+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.887+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.888+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.890+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.891+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.893+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.894+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.896+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.897+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.899+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.900+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.009 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.902+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.903+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.905+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.907+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.908+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.910+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.911+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.913+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.914+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.916+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.917+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.010 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.919+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.921+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.922+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.924+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.925+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.927+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.928+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.930+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.931+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.933+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.934+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.936+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.937+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.011 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.939+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.940+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.942+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.943+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.945+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.946+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.948+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.949+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.951+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.953+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.954+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.956+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.957+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.012 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.959+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.960+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.962+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.963+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.965+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.966+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.968+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.970+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.971+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.973+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.974+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.976+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.977+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.013 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.979+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.980+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.982+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.983+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.985+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.986+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.988+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.989+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.991+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.992+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.994+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:52.996599] tcp.c: 659:nvmf_tcp_create: *NOTICE*: *** TCP Transport Init *** 2024-05-21T12:39:53.014 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.995+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.015 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.997+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.015 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:52.998+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.015 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.000+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.015 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.001+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.015 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.003+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.106 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/448261625' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:39:53.107 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.863895] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:39:53.107 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 2024-05-21T12:39:53.107 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:53.107 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.107 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:39:53.107 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.885144] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:39:53.107 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 2024-05-21T12:39:53.107 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:53.108 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.108 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:39:53.108 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.907164] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:39:53.108 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 2024-05-21T12:39:53.108 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.108 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.108 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:39:53.109 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.927939] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:39:53.109 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 2024-05-21T12:39:53.109 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:53.109 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.109 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:39:53.109 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.950724] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:39:53.109 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da 2024-05-21T12:39:53.109 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.972507] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:52.993297] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:52 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:52] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.110 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.015773] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.039269] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.064942] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.111 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.112 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:39:53.112 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.086106] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:39:53.112 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 2024-05-21T12:39:53.112 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.112 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.112 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:39:53.112 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.107047] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:39:53.112 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 2024-05-21T12:39:53.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.005+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.006+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.008+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO server.py:268: Discovery service process id: 873 2024-05-21T12:39:53.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO server.py:264: Starting ceph nvmeof discovery service 2024-05-21T12:39:53.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.009+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.011+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.013+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.015+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.016+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.018+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.020+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.022+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO cephutils.py:74: Connected to Ceph with version "19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)" 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:86: Requested huge pages count is 2048 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:105: Actual huge pages count is 4096 2024-05-21T12:39:53.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:190: NVMeoF bdevs per cluster: 32 2024-05-21T12:39:53.255 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.024+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:39:53.255 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:39:53.255 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO discovery.py:322: log pages info from omap: nvmeof.state 2024-05-21T12:39:53.255 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO server.py:195: Prometheus endpoint is enabled 2024-05-21T12:39:53.255 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:39:53.255 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.255 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 1 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 2 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 3 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.256 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 4 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 5 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 6 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 7 2024-05-21T12:39:53.257 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 8 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 9 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 10 2024-05-21T12:39:53.258 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 11 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 12 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.259 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 13 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 14 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 15 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 16 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.260 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.261 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:39:53.261 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.261 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 1 2024-05-21T12:39:53.261 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.261 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.261 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.261 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 2 2024-05-21T12:39:53.261 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.262 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.262 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.262 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 3 2024-05-21T12:39:53.262 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 4 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 5 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 6 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.263 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 7 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 8 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 9 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.264 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 10 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 11 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 12 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 13 2024-05-21T12:39:53.265 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 14 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 15 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 16 2024-05-21T12:39:53.266 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 1 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 2 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 3 2024-05-21T12:39:53.267 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 4 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 5 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 6 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.268 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 7 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 8 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 9 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.269 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 10 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 11 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 12 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.270 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 13 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 14 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 15 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 16 2024-05-21T12:39:53.271 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: INACCESSIBLE 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: ], , client address: IPv4 172.21.15.92:49080 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode1 not found 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode2 not found 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode3 not found 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO prometheus.py:98: Prometheus exporter running in http mode, listening on port 10008 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:2648: Received request to get gateway's info 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.034+0000 7fea13e56640 0 ms_deliver_dispatch: unhandled message 0x55facd725180 nvmeofgwmap magic: 0 from mon.2 v2:172.21.15.72:3300/0 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.034+0000 7fea13e56640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 4 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.034+0000 7fea13e56640 0 ms_deliver_dispatch: unhandled message 0x55facd715d40 mon_map magic: 0 from mon.2 v2:172.21.15.72:3300/0 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.034+0000 7fea13e56640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 41 2024-05-21T12:39:53.272 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.035+0000 7fea13e56640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.035+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 137 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 137 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,OPTIMIZED ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 137 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.035+0000 7fea13e56640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) got_old_gw_state: NvmeGwState { group id: 2 gw_map_epoch 136 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }got_new_gw_state: NvmeGwState { group id: 2 gw_map_epoch 137 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: 2024-05-21T12:39:53.035+0000 7fea13e56640 0 ms_deliver_dispatch: unhandled message 0x55facd725500 nvmeofgwmap magic: 0 from mon.2 v2:172.21.15.72:3300/0 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO prometheus.py:131: Stats for all bdevs will be provided 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO discovery.py:330: discovery addr: 172.21.15.92 port: 8009 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.080585] nvmf_rpc.c: 275:rpc_nvmf_get_subsystems: *WARNING*: rpc_nvmf_get_subsystems: deprecated feature listener.transport is deprecated in favor of trtype to be removed in v24.05 2024-05-21T12:39:53.273 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.113562] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.92:0/2249250809, rc 0 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.114103] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.92:0/2249250809 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.92:0/2249250809' anagrp=1 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.127929] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:39:53.274 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.174192] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.218341] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.92:0/3388488487, rc 0 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.218452] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.92:0/3388488487 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.92:0/3388488487' anagrp=2 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.233550] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.275 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:39:53.335 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:52 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/448261625' entity='client.nvmeof.nvmeof.a' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:39:53.364 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.364 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.364 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:39:53.364 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.128194] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:39:53.364 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 2024-05-21T12:39:53.364 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:53.364 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.364 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:39:53.365 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.149459] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:39:53.365 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b 2024-05-21T12:39:53.365 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.365 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.365 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:39:53.365 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.172454] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:39:53.365 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 2024-05-21T12:39:53.366 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.366 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.366 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:39:53.366 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.196036] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:39:53.366 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 2024-05-21T12:39:53.366 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.218629] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.241053] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.367 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.319499] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.341753] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.364383] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:39:53.368 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:39:53.512 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.260378] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:39:53.512 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.318907] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.353167] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.92:0/790513900, rc 0 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.353458] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.92:0/790513900 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.92:0/790513900' anagrp=3 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:39:53.513 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.367676] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.390839] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.411962] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:39:53.514 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b 2024-05-21T12:39:53.515 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.515 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.515 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:39:53.515 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.433634] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:39:53.515 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a 2024-05-21T12:39:53.515 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.458864] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.481657] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.516 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:39:53.517 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.503225] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:39:53.517 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 2024-05-21T12:39:53.517 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.517 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.517 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:39:53.615 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:53.615 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.615 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:39:53.615 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.384951] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:39:53.615 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 2024-05-21T12:39:53.615 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:53.615 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.615 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.407842] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.431209] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.454186] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:39:53.616 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.475081] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.498186] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:39:53.617 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 2024-05-21T12:39:53.618 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.618 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.618 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:39:53.618 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.520321] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:39:53.618 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 2024-05-21T12:39:53.618 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:53.618 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.541682] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.564437] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.585239] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:39:53.619 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 2024-05-21T12:39:53.620 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.620 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.620 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:39:53.620 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.606208] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:39:53.620 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca 2024-05-21T12:39:53.620 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:53.620 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.620 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:39:53.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.526620] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:39:53.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d 2024-05-21T12:39:53.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:53.769 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.552375] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.573687] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:39:53.770 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.594361] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.615606] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.638444] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:53.771 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:39:53.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.659946] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:39:53.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea 2024-05-21T12:39:53.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:53.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.772 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.681422] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.702498] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.724548] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 2024-05-21T12:39:53.773 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.745581] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.768886] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:39:53.774 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc 2024-05-21T12:39:53.868 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.627405] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:39:53.868 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.649208] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.671742] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.869 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:39:53.870 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.693386] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:39:53.870 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b 2024-05-21T12:39:53.870 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.714120] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.737563] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:39:53.871 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.762576] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:39:53.872 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d 2024-05-21T12:39:53.872 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:53.872 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.872 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:39:53.872 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.783370] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:39:53.873 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd 2024-05-21T12:39:53.873 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.873 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.873 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:39:53.873 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.805054] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:39:53.873 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a 2024-05-21T12:39:53.873 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:53.873 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.874 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:39:53.874 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.826600] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:39:53.874 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 2024-05-21T12:39:53.875 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:53.875 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.875 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:39:53.875 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.847125] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:39:53.875 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed 2024-05-21T12:39:53.875 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:53.875 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:53.875 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:39:53.876 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.868375] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:39:53.876 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a 2024-05-21T12:39:54.029 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-mon[33665]: pgmap v810: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 8.4 MiB/s wr, 589 op/s 2024-05-21T12:39:54.029 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/2976214607' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.791152] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:39:54.030 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.812064] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:39:54.031 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b 2024-05-21T12:39:54.031 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:54.031 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.031 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:39:54.031 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.832838] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:39:54.031 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.854990] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.877471] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:39:54.032 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.899166] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.921017] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:39:54.033 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.943150] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.965914] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 2024-05-21T12:39:54.034 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:53.986311] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.008858] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:39:54.035 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c 2024-05-21T12:39:54.036 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:54.036 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.036 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:39:54.036 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.029337] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:39:54.036 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 2024-05-21T12:39:54.286 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:54.286 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.286 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:39:54.286 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.053255] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.073204] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.094691] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b 2024-05-21T12:39:54.287 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.116363] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.139545] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:54.288 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.160795] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.181634] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.202574] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:39:54.289 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 2024-05-21T12:39:54.290 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:54.290 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.290 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:39:54.290 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.224669] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:39:54.290 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f 2024-05-21T12:39:54.290 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:54.290 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.290 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.244552] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.265518] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.285715] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:39:54.291 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 2024-05-21T12:39:54.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-mon[30165]: pgmap v810: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 8.4 MiB/s wr, 589 op/s 2024-05-21T12:39:54.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/2976214607' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.889856] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.920568] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea 2024-05-21T12:39:54.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:53.957264] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.45 port 4420 *** 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: None 2024-05-21T12:39:54.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:53 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:53] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: None 2024-05-21T12:39:54.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-mon[33912]: pgmap v810: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.4 MiB/s rd, 8.4 MiB/s wr, 589 op/s 2024-05-21T12:39:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/2976214607' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:39:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:39:53] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:39:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:39:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:39:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:39:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:39:54.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 1 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:39:54.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:53 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:39862 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.306454] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.330617] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:39:54.540 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.354740] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.377982] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.541 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:39:54.542 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.400805] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:39:54.542 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 2024-05-21T12:39:54.542 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:39:54.542 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.542 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:39:54.542 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.420582] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:39:54.542 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.440911] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.464203] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:39:54.543 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.484828] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.507730] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.544 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:39:54.545 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.530709] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:39:54.545 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 2024-05-21T12:39:54.545 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:39:54.545 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:39:54.545 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:39:54.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:39:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 94.0k 119.6k 0.0k 940.0k 1.2M 0.0k nvme2n1 2024-05-21T12:39:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 97.2k 107.2k 0.0k 972.0k 1.0M 0.0k nvme2n10 2024-05-21T12:39:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 122.0k 85.2k 0.0k 1.2M 852.0k 0.0k nvme2n11 2024-05-21T12:39:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 110.8k 125.2k 0.0k 1.1M 1.2M 0.0k nvme2n12 2024-05-21T12:39:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 132.0k 99.2k 0.0k 1.3M 992.0k 0.0k nvme2n13 2024-05-21T12:39:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 112.0k 142.0k 0.0k 1.1M 1.4M 0.0k nvme2n14 2024-05-21T12:39:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 126.4k 136.0k 0.0k 1.2M 1.3M 0.0k nvme2n15 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 132.8k 132.4k 0.0k 1.3M 1.3M 0.0k nvme2n16 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 116.4k 118.8k 0.0k 1.1M 1.2M 0.0k nvme2n17 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 116.4k 116.8k 0.0k 1.1M 1.1M 0.0k nvme2n18 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 142.0k 103.6k 0.0k 1.4M 1.0M 0.0k nvme2n19 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 125.2k 103.2k 0.0k 1.2M 1.0M 0.0k nvme2n2 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 82.8k 129.2k 0.0k 828.0k 1.3M 0.0k nvme2n20 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 114.0k 100.8k 0.0k 1.1M 1008.0k 0.0k nvme2n3 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 93.2k 106.4k 0.0k 932.0k 1.0M 0.0k nvme2n4 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 113.2k 115.6k 0.0k 1.1M 1.1M 0.0k nvme2n5 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 83.6k 130.0k 0.0k 836.0k 1.3M 0.0k nvme2n6 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 141.2k 110.8k 0.0k 1.4M 1.1M 0.0k nvme2n7 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 140.4k 95.6k 0.0k 1.4M 956.0k 0.0k nvme2n8 2024-05-21T12:39:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 106.0k 103.6k 0.0k 1.0M 1.0M 0.0k nvme2n9 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 127.6k 94.8k 0.0k 1.2M 948.0k 0.0k nvme5n1 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 115.6k 136.8k 0.0k 1.1M 1.3M 0.0k nvme5n10 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 124.0k 105.2k 0.0k 1.2M 1.0M 0.0k nvme5n11 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 106.0k 142.8k 0.0k 1.0M 1.4M 0.0k nvme5n12 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 121.2k 126.4k 0.0k 1.2M 1.2M 0.0k nvme5n13 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 111.2k 139.2k 0.0k 1.1M 1.4M 0.0k nvme5n14 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 108.8k 130.8k 0.0k 1.1M 1.3M 0.0k nvme5n15 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 100.0k 101.2k 0.0k 1000.0k 1012.0k 0.0k nvme5n16 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 95.6k 124.8k 0.0k 956.0k 1.2M 0.0k nvme5n17 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 114.4k 138.8k 0.0k 1.1M 1.4M 0.0k nvme5n18 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 124.0k 120.4k 0.0k 1.2M 1.2M 0.0k nvme5n19 2024-05-21T12:39:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 89.2k 140.8k 0.0k 892.0k 1.4M 0.0k nvme5n2 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 74.8k 146.0k 0.0k 748.0k 1.4M 0.0k nvme5n20 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 160.0k 136.0k 0.0k 1.6M 1.3M 0.0k nvme5n3 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 96.8k 150.0k 0.0k 968.0k 1.5M 0.0k nvme5n4 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 160.8k 70.4k 0.0k 1.6M 704.0k 0.0k nvme5n5 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 112.8k 107.6k 0.0k 1.1M 1.1M 0.0k nvme5n6 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 124.0k 89.6k 0.0k 1.2M 896.0k 0.0k nvme5n7 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 111.2k 108.8k 0.0k 1.1M 1.1M 0.0k nvme5n8 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 94.4k 91.2k 0.0k 944.0k 912.0k 0.0k nvme5n9 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 146.4k 123.6k 0.0k 1.4M 1.2M 0.0k nvme8n1 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 121.2k 125.2k 0.0k 1.2M 1.2M 0.0k nvme8n10 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 104.0k 136.8k 0.0k 1.0M 1.3M 0.0k nvme8n11 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 103.6k 106.4k 0.0k 1.0M 1.0M 0.0k nvme8n12 2024-05-21T12:39:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 123.2k 125.6k 0.0k 1.2M 1.2M 0.0k nvme8n13 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 114.0k 112.0k 0.0k 1.1M 1.1M 0.0k nvme8n14 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 120.4k 144.0k 0.0k 1.2M 1.4M 0.0k nvme8n15 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 119.2k 111.6k 0.0k 1.2M 1.1M 0.0k nvme8n16 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 95.2k 133.2k 0.0k 952.0k 1.3M 0.0k nvme8n17 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 142.4k 111.6k 0.0k 1.4M 1.1M 0.0k nvme8n18 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 129.2k 108.8k 0.0k 1.3M 1.1M 0.0k nvme8n19 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 109.2k 123.2k 0.0k 1.1M 1.2M 0.0k nvme8n2 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 98.0k 118.8k 0.0k 980.0k 1.2M 0.0k nvme8n20 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 105.2k 157.6k 0.0k 1.0M 1.5M 0.0k nvme8n3 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 154.8k 106.0k 0.0k 1.5M 1.0M 0.0k nvme8n4 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 112.8k 122.0k 0.0k 1.1M 1.2M 0.0k nvme8n5 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 136.4k 120.8k 0.0k 1.3M 1.2M 0.0k nvme8n6 2024-05-21T12:39:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 104.8k 114.0k 0.0k 1.0M 1.1M 0.0k nvme8n7 2024-05-21T12:39:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 119.2k 114.4k 0.0k 1.2M 1.1M 0.0k nvme8n8 2024-05-21T12:39:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 150.0k 97.2k 0.0k 1.5M 972.0k 0.0k nvme8n9 2024-05-21T12:39:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:39:54.957 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.553720] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:39:54.957 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:54.598170] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.92 port 4420 *** 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: None 2024-05-21T12:39:54.958 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: None 2024-05-21T12:39:54.959 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:54] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: None 2024-05-21T12:39:55.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:54 smithi092 ceph-mon[33665]: pgmap v811: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 557 op/s 2024-05-21T12:39:55.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:54 smithi045 ceph-mon[30165]: pgmap v811: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 557 op/s 2024-05-21T12:39:55.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-mon[33912]: pgmap v811: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 557 op/s 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:39:54] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:39:55.416 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:39:55.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:39:55.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:39:55.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:39:55.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:39:55.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:39:55.417 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:51082 2024-05-21T12:39:57.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:56 smithi092 ceph-mon[33665]: pgmap v812: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 775 op/s 2024-05-21T12:39:57.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:56 smithi045 ceph-mon[30165]: pgmap v812: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 775 op/s 2024-05-21T12:39:57.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:56 smithi072 ceph-mon[33912]: pgmap v812: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 8.0 MiB/s rd, 7.4 MiB/s wr, 775 op/s 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:39:57] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 3 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: OPTIMIZED 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 3 2024-05-21T12:39:58.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: OPTIMIZED 2024-05-21T12:39:58.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:58.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:39:58.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:39:58.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 3 2024-05-21T12:39:58.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: OPTIMIZED 2024-05-21T12:39:58.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:39:58.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: ], , client address: IPv4 172.21.15.92:37124 2024-05-21T12:39:58.254 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:57.967197] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000285 seconds 2024-05-21T12:39:58.316 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:39:57] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 1 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 1 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:39:58.317 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 1 2024-05-21T12:39:58.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:39:58.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:39:58.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: ], , client address: IPv4 172.21.15.45:51938 2024-05-21T12:39:58.318 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:57.966555] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000228 seconds 2024-05-21T12:39:59.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:58 smithi092 ceph-mon[33665]: pgmap v813: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 5.6 MiB/s wr, 726 op/s 2024-05-21T12:39:59.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:58 smithi045 ceph-mon[30165]: pgmap v813: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 5.6 MiB/s wr, 726 op/s 2024-05-21T12:39:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:58 smithi072 ceph-mon[33912]: pgmap v813: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 5.6 MiB/s wr, 726 op/s 2024-05-21T12:40:00.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:40:00.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:40:00.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:40:00.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:59 smithi092 ceph-mon[33665]: pgmap v814: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 5.3 MiB/s wr, 774 op/s 2024-05-21T12:40:00.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:59 smithi092 ceph-mon[33665]: pgmap v815: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 5.1 MiB/s wr, 845 op/s 2024-05-21T12:40:00.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:39:59 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:40:00.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:40:00.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:40:00.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:40:00.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:59 smithi045 ceph-mon[30165]: pgmap v814: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 5.3 MiB/s wr, 774 op/s 2024-05-21T12:40:00.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:59 smithi045 ceph-mon[30165]: pgmap v815: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 5.1 MiB/s wr, 845 op/s 2024-05-21T12:40:00.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:39:59 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:40:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:40:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:40:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:40:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:59 smithi072 ceph-mon[33912]: pgmap v814: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 5.3 MiB/s wr, 774 op/s 2024-05-21T12:40:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:59 smithi072 ceph-mon[33912]: pgmap v815: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.0 MiB/s rd, 5.1 MiB/s wr, 845 op/s 2024-05-21T12:40:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:39:59 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:40:01.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:00 smithi092 ceph-mon[33665]: Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:40:01.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:00 smithi092 ceph-mon[33665]: [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:40:01.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:00 smithi092 ceph-mon[33665]: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:40:01.253 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:00 smithi092 ceph-mon[33665]: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:40:01.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:00 smithi045 ceph-mon[30165]: Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:40:01.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:00 smithi045 ceph-mon[30165]: [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:40:01.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:00 smithi045 ceph-mon[30165]: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:40:01.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:00 smithi045 ceph-mon[30165]: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:40:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:00 smithi072 ceph-mon[33912]: Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:40:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:00 smithi072 ceph-mon[33912]: [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:40:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:00 smithi072 ceph-mon[33912]: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:40:01.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:00 smithi072 ceph-mon[33912]: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:40:02.252 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:01 smithi092 ceph-mon[33665]: pgmap v816: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.5 MiB/s wr, 804 op/s 2024-05-21T12:40:02.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:01 smithi045 ceph-mon[30165]: pgmap v816: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.5 MiB/s wr, 804 op/s 2024-05-21T12:40:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:01 smithi072 ceph-mon[33912]: pgmap v816: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 3.5 MiB/s wr, 804 op/s 2024-05-21T12:40:04.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:04 smithi045 ceph-mon[30165]: pgmap v817: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 3.6 MiB/s wr, 783 op/s 2024-05-21T12:40:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:04 smithi072 ceph-mon[33912]: pgmap v817: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 3.6 MiB/s wr, 783 op/s 2024-05-21T12:40:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:04 smithi092 ceph-mon[33665]: pgmap v817: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 3.6 MiB/s wr, 783 op/s 2024-05-21T12:40:04.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:04.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.80 124.8k 58.0k 0.0k 1.2M 580.0k 0.0k nvme2n1 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 92.0k 84.4k 0.0k 920.0k 844.0k 0.0k nvme2n10 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.10 130.8k 80.8k 0.0k 1.3M 808.0k 0.0k nvme2n11 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 102.4k 74.8k 0.0k 1.0M 748.0k 0.0k nvme2n12 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 111.2k 72.8k 0.0k 1.1M 728.0k 0.0k nvme2n13 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.00 92.8k 78.8k 0.0k 928.0k 788.0k 0.0k nvme2n14 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.90 88.4k 92.0k 0.0k 884.0k 920.0k 0.0k nvme2n15 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 114.0k 59.6k 0.0k 1.1M 596.0k 0.0k nvme2n16 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 6.80 105.2k 84.0k 0.0k 1.0M 840.0k 0.0k nvme2n17 2024-05-21T12:40:04.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 114.0k 92.4k 0.0k 1.1M 924.0k 0.0k nvme2n18 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.00 114.8k 64.0k 0.0k 1.1M 640.0k 0.0k nvme2n19 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 93.6k 87.6k 0.0k 936.0k 876.0k 0.0k nvme2n2 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 125.2k 52.4k 0.0k 1.2M 524.0k 0.0k nvme2n20 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.00 95.2k 78.0k 0.0k 952.0k 780.0k 0.0k nvme2n3 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 81.6k 75.2k 0.0k 816.0k 752.0k 0.0k nvme2n4 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 112.8k 76.8k 0.0k 1.1M 768.0k 0.0k nvme2n5 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 112.0k 75.6k 0.0k 1.1M 756.0k 0.0k nvme2n6 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 105.2k 73.6k 0.0k 1.0M 736.0k 0.0k nvme2n7 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 118.4k 78.8k 0.0k 1.2M 788.0k 0.0k nvme2n8 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.40 140.4k 68.0k 0.0k 1.4M 680.0k 0.0k nvme2n9 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 6.70 93.2k 69.2k 0.0k 932.0k 692.0k 0.0k nvme5n1 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 104.0k 79.2k 0.0k 1.0M 792.0k 0.0k nvme5n10 2024-05-21T12:40:04.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.10 118.4k 71.2k 0.0k 1.2M 712.0k 0.0k nvme5n11 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 115.2k 86.8k 0.0k 1.1M 868.0k 0.0k nvme5n12 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 117.6k 57.2k 0.0k 1.1M 572.0k 0.0k nvme5n13 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 115.6k 70.8k 0.0k 1.1M 708.0k 0.0k nvme5n14 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 111.2k 61.6k 0.0k 1.1M 616.0k 0.0k nvme5n15 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.80 131.2k 47.2k 0.0k 1.3M 472.0k 0.0k nvme5n16 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.00 88.4k 51.6k 0.0k 884.0k 516.0k 0.0k nvme5n17 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.00 106.8k 78.8k 0.0k 1.0M 788.0k 0.0k nvme5n18 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 126.0k 64.4k 0.0k 1.2M 644.0k 0.0k nvme5n19 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 96.8k 60.4k 0.0k 968.0k 604.0k 0.0k nvme5n2 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 6.30 120.4k 83.6k 0.0k 1.2M 836.0k 0.0k nvme5n20 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.10 139.2k 82.8k 0.0k 1.4M 828.0k 0.0k nvme5n3 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.00 130.0k 69.2k 0.0k 1.3M 692.0k 0.0k nvme5n4 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 98.4k 73.6k 0.0k 984.0k 736.0k 0.0k nvme5n5 2024-05-21T12:40:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.70 115.2k 90.0k 0.0k 1.1M 900.0k 0.0k nvme5n6 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.90 110.0k 71.6k 0.0k 1.1M 716.0k 0.0k nvme5n7 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 113.2k 66.0k 0.0k 1.1M 660.0k 0.0k nvme5n8 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 108.8k 50.0k 0.0k 1.1M 500.0k 0.0k nvme5n9 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.40 100.4k 94.8k 0.0k 1004.0k 948.0k 0.0k nvme8n1 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 90.0k 61.2k 0.0k 900.0k 612.0k 0.0k nvme8n10 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.00 104.0k 72.0k 0.0k 1.0M 720.0k 0.0k nvme8n11 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 119.2k 66.4k 0.0k 1.2M 664.0k 0.0k nvme8n12 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.40 116.0k 98.4k 0.0k 1.1M 984.0k 0.0k nvme8n13 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.40 114.4k 68.0k 0.0k 1.1M 680.0k 0.0k nvme8n14 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.50 110.0k 74.8k 0.0k 1.1M 748.0k 0.0k nvme8n15 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.10 118.4k 28.4k 0.0k 1.2M 284.0k 0.0k nvme8n16 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.50 112.4k 85.6k 0.0k 1.1M 856.0k 0.0k nvme8n17 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.60 97.2k 98.0k 0.0k 972.0k 980.0k 0.0k nvme8n18 2024-05-21T12:40:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 6.70 99.6k 91.2k 0.0k 996.0k 912.0k 0.0k nvme8n19 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.10 136.0k 52.0k 0.0k 1.3M 520.0k 0.0k nvme8n2 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.10 146.0k 44.0k 0.0k 1.4M 440.0k 0.0k nvme8n20 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 6.80 129.2k 66.4k 0.0k 1.3M 664.0k 0.0k nvme8n3 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 6.50 86.4k 90.4k 0.0k 864.0k 904.0k 0.0k nvme8n4 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 132.0k 67.6k 0.0k 1.3M 676.0k 0.0k nvme8n5 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 102.8k 70.4k 0.0k 1.0M 704.0k 0.0k nvme8n6 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 6.90 107.6k 71.6k 0.0k 1.1M 716.0k 0.0k nvme8n7 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 6.60 97.6k 65.6k 0.0k 976.0k 656.0k 0.0k nvme8n8 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 138.4k 57.6k 0.0k 1.4M 576.0k 0.0k nvme8n9 2024-05-21T12:40:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:06.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:06 smithi045 ceph-mon[30165]: pgmap v818: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 519 op/s 2024-05-21T12:40:06.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:06 smithi072 ceph-mon[33912]: pgmap v818: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 519 op/s 2024-05-21T12:40:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:06 smithi092 ceph-mon[33665]: pgmap v818: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 3.6 MiB/s wr, 519 op/s 2024-05-21T12:40:08.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:08 smithi045 ceph-mon[30165]: pgmap v819: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-05-21T12:40:08.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:08 smithi072 ceph-mon[33912]: pgmap v819: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-05-21T12:40:08.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:08 smithi092 ceph-mon[33665]: pgmap v819: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-05-21T12:40:10.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:10 smithi045 ceph-mon[30165]: pgmap v820: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 5.9 MiB/s wr, 554 op/s 2024-05-21T12:40:10.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:10 smithi072 ceph-mon[33912]: pgmap v820: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 5.9 MiB/s wr, 554 op/s 2024-05-21T12:40:10.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:10 smithi092 ceph-mon[33665]: pgmap v820: 33 pgs: 33 active+clean; 1.0 GiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 5.9 MiB/s wr, 554 op/s 2024-05-21T12:40:12.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:12 smithi045 ceph-mon[30165]: pgmap v821: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 555 op/s 2024-05-21T12:40:12.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:12 smithi072 ceph-mon[33912]: pgmap v821: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 555 op/s 2024-05-21T12:40:12.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:12 smithi092 ceph-mon[33665]: pgmap v821: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 555 op/s 2024-05-21T12:40:14.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:14 smithi045 ceph-mon[30165]: pgmap v822: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 503 op/s 2024-05-21T12:40:14.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:14 smithi072 ceph-mon[33912]: pgmap v822: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 503 op/s 2024-05-21T12:40:14.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:14 smithi092 ceph-mon[33665]: pgmap v822: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.2 MiB/s wr, 503 op/s 2024-05-21T12:40:14.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 94.8k 105.6k 0.0k 948.0k 1.0M 0.0k nvme2n1 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.0k 97.6k 0.0k 1.1M 976.0k 0.0k nvme2n10 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.8k 136.8k 0.0k 1.1M 1.3M 0.0k nvme2n11 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.4k 98.4k 0.0k 1.1M 984.0k 0.0k nvme2n12 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 93.6k 94.4k 0.0k 936.0k 944.0k 0.0k nvme2n13 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 107.6k 117.6k 0.0k 1.1M 1.1M 0.0k nvme2n14 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 72.8k 146.4k 0.0k 728.0k 1.4M 0.0k nvme2n15 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 125.6k 106.0k 0.0k 1.2M 1.0M 0.0k nvme2n16 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 130.4k 103.2k 0.0k 1.3M 1.0M 0.0k nvme2n17 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 115.6k 96.0k 0.0k 1.1M 960.0k 0.0k nvme2n18 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 103.2k 96.8k 0.0k 1.0M 968.0k 0.0k nvme2n19 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 127.6k 97.2k 0.0k 1.2M 972.0k 0.0k nvme2n2 2024-05-21T12:40:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 70.0k 115.6k 0.0k 700.0k 1.1M 0.0k nvme2n20 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 120.4k 111.2k 0.0k 1.2M 1.1M 0.0k nvme2n3 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 89.2k 113.6k 0.0k 892.0k 1.1M 0.0k nvme2n4 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 93.6k 121.2k 0.0k 936.0k 1.2M 0.0k nvme2n5 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 121.2k 68.0k 0.0k 1.2M 680.0k 0.0k nvme2n6 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 95.2k 103.2k 0.0k 952.0k 1.0M 0.0k nvme2n7 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 120.0k 139.6k 0.0k 1.2M 1.4M 0.0k nvme2n8 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.0k 100.8k 0.0k 1.1M 1008.0k 0.0k nvme2n9 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 95.2k 112.0k 0.0k 952.0k 1.1M 0.0k nvme5n1 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 118.0k 104.8k 0.0k 1.2M 1.0M 0.0k nvme5n10 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 88.0k 108.0k 0.0k 880.0k 1.1M 0.0k nvme5n11 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 113.6k 143.2k 0.0k 1.1M 1.4M 0.0k nvme5n12 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 113.2k 113.2k 0.0k 1.1M 1.1M 0.0k nvme5n13 2024-05-21T12:40:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 89.2k 127.2k 0.0k 892.0k 1.2M 0.0k nvme5n14 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 138.8k 105.6k 0.0k 1.4M 1.0M 0.0k nvme5n15 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 71.6k 102.4k 0.0k 716.0k 1.0M 0.0k nvme5n16 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 84.4k 107.2k 0.0k 844.0k 1.0M 0.0k nvme5n17 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 102.4k 116.4k 0.0k 1.0M 1.1M 0.0k nvme5n18 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 115.2k 97.2k 0.0k 1.1M 972.0k 0.0k nvme5n19 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 90.0k 112.0k 0.0k 900.0k 1.1M 0.0k nvme5n2 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 126.4k 102.4k 0.0k 1.2M 1.0M 0.0k nvme5n20 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 135.6k 113.2k 0.0k 1.3M 1.1M 0.0k nvme5n3 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 106.0k 119.6k 0.0k 1.0M 1.2M 0.0k nvme5n4 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 103.2k 101.6k 0.0k 1.0M 1016.0k 0.0k nvme5n5 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 98.4k 139.2k 0.0k 984.0k 1.4M 0.0k nvme5n6 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 104.4k 88.4k 0.0k 1.0M 884.0k 0.0k nvme5n7 2024-05-21T12:40:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 76.8k 135.2k 0.0k 768.0k 1.3M 0.0k nvme5n8 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 101.2k 110.8k 0.0k 1012.0k 1.1M 0.0k nvme5n9 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 127.2k 103.2k 0.0k 1.2M 1.0M 0.0k nvme8n1 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.4k 101.2k 0.0k 1.1M 1012.0k 0.0k nvme8n10 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 111.6k 106.8k 0.0k 1.1M 1.0M 0.0k nvme8n11 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 98.4k 123.2k 0.0k 984.0k 1.2M 0.0k nvme8n12 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.4k 92.4k 0.0k 1.1M 924.0k 0.0k nvme8n13 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 93.6k 108.8k 0.0k 936.0k 1.1M 0.0k nvme8n14 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 115.6k 105.6k 0.0k 1.1M 1.0M 0.0k nvme8n15 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 109.2k 80.0k 0.0k 1.1M 800.0k 0.0k nvme8n16 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 109.6k 114.0k 0.0k 1.1M 1.1M 0.0k nvme8n17 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 116.8k 107.6k 0.0k 1.1M 1.1M 0.0k nvme8n18 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 123.2k 73.2k 0.0k 1.2M 732.0k 0.0k nvme8n19 2024-05-21T12:40:14.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 106.0k 64.0k 0.0k 1.0M 640.0k 0.0k nvme8n2 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 97.6k 89.6k 0.0k 976.0k 896.0k 0.0k nvme8n20 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 113.6k 98.8k 0.0k 1.1M 988.0k 0.0k nvme8n3 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 101.6k 111.2k 0.0k 1016.0k 1.1M 0.0k nvme8n4 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 123.2k 90.0k 0.0k 1.2M 900.0k 0.0k nvme8n5 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 92.4k 118.4k 0.0k 924.0k 1.2M 0.0k nvme8n6 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 85.6k 114.8k 0.0k 856.0k 1.1M 0.0k nvme8n7 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 107.2k 97.6k 0.0k 1.0M 976.0k 0.0k nvme8n8 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 102.8k 126.0k 0.0k 1.0M 1.2M 0.0k nvme8n9 2024-05-21T12:40:14.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:16.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:16 smithi045 ceph-mon[30165]: pgmap v823: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 487 op/s 2024-05-21T12:40:16.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:16 smithi072 ceph-mon[33912]: pgmap v823: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 487 op/s 2024-05-21T12:40:16.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:16 smithi092 ceph-mon[33665]: pgmap v823: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 487 op/s 2024-05-21T12:40:18.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:18 smithi045 ceph-mon[30165]: pgmap v824: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 547 op/s 2024-05-21T12:40:18.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:18 smithi072 ceph-mon[33912]: pgmap v824: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 547 op/s 2024-05-21T12:40:18.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:18 smithi092 ceph-mon[33665]: pgmap v824: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 547 op/s 2024-05-21T12:40:20.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:20 smithi045 ceph-mon[30165]: pgmap v825: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 502 op/s 2024-05-21T12:40:20.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:20 smithi072 ceph-mon[33912]: pgmap v825: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 502 op/s 2024-05-21T12:40:20.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:20 smithi092 ceph-mon[33665]: pgmap v825: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 502 op/s 2024-05-21T12:40:22.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:22 smithi045 ceph-mon[30165]: pgmap v826: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 519 op/s 2024-05-21T12:40:22.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:22 smithi072 ceph-mon[33912]: pgmap v826: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 519 op/s 2024-05-21T12:40:22.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:22 smithi092 ceph-mon[33665]: pgmap v826: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 519 op/s 2024-05-21T12:40:24.304 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:24 smithi092 ceph-mon[33665]: pgmap v827: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 485 op/s 2024-05-21T12:40:24.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:24 smithi045 ceph-mon[30165]: pgmap v827: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 485 op/s 2024-05-21T12:40:24.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:24 smithi072 ceph-mon[33912]: pgmap v827: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 485 op/s 2024-05-21T12:40:24.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 105.6k 103.6k 0.0k 1.0M 1.0M 0.0k nvme2n1 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 106.4k 112.4k 0.0k 1.0M 1.1M 0.0k nvme2n10 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 108.4k 118.4k 0.0k 1.1M 1.2M 0.0k nvme2n11 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 114.4k 93.6k 0.0k 1.1M 936.0k 0.0k nvme2n12 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 96.0k 101.2k 0.0k 960.0k 1012.0k 0.0k nvme2n13 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 116.8k 76.4k 0.0k 1.1M 764.0k 0.0k nvme2n14 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 99.6k 96.0k 0.0k 996.0k 960.0k 0.0k nvme2n15 2024-05-21T12:40:24.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 80.0k 100.4k 0.0k 800.0k 1004.0k 0.0k nvme2n16 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 81.2k 132.4k 0.0k 812.0k 1.3M 0.0k nvme2n17 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 99.6k 84.4k 0.0k 996.0k 844.0k 0.0k nvme2n18 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 95.6k 96.4k 0.0k 956.0k 964.0k 0.0k nvme2n19 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 96.0k 100.0k 0.0k 960.0k 1000.0k 0.0k nvme2n2 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 122.0k 72.8k 0.0k 1.2M 728.0k 0.0k nvme2n20 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 106.8k 102.8k 0.0k 1.0M 1.0M 0.0k nvme2n3 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 85.2k 93.6k 0.0k 852.0k 936.0k 0.0k nvme2n4 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 98.8k 102.4k 0.0k 988.0k 1.0M 0.0k nvme2n5 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 109.6k 72.4k 0.0k 1.1M 724.0k 0.0k nvme2n6 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 89.6k 110.0k 0.0k 896.0k 1.1M 0.0k nvme2n7 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 116.0k 104.4k 0.0k 1.1M 1.0M 0.0k nvme2n8 2024-05-21T12:40:24.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 101.2k 95.6k 0.0k 1012.0k 956.0k 0.0k nvme2n9 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 70.8k 130.8k 0.0k 708.0k 1.3M 0.0k nvme5n1 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 96.8k 103.6k 0.0k 968.0k 1.0M 0.0k nvme5n10 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 101.2k 94.8k 0.0k 1012.0k 948.0k 0.0k nvme5n11 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 88.0k 124.4k 0.0k 880.0k 1.2M 0.0k nvme5n12 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 106.0k 118.4k 0.0k 1.0M 1.2M 0.0k nvme5n13 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 114.4k 131.2k 0.0k 1.1M 1.3M 0.0k nvme5n14 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 117.6k 94.4k 0.0k 1.1M 944.0k 0.0k nvme5n15 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 87.2k 102.8k 0.0k 872.0k 1.0M 0.0k nvme5n16 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 88.8k 110.4k 0.0k 888.0k 1.1M 0.0k nvme5n17 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 91.2k 115.6k 0.0k 912.0k 1.1M 0.0k nvme5n18 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 106.4k 108.0k 0.0k 1.0M 1.1M 0.0k nvme5n19 2024-05-21T12:40:24.887 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 92.8k 113.2k 0.0k 928.0k 1.1M 0.0k nvme5n2 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 97.6k 99.2k 0.0k 976.0k 992.0k 0.0k nvme5n20 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 114.4k 98.8k 0.0k 1.1M 988.0k 0.0k nvme5n3 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 114.4k 109.6k 0.0k 1.1M 1.1M 0.0k nvme5n4 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 78.0k 110.8k 0.0k 780.0k 1.1M 0.0k nvme5n5 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 98.4k 118.4k 0.0k 984.0k 1.2M 0.0k nvme5n6 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 102.0k 104.0k 0.0k 1020.0k 1.0M 0.0k nvme5n7 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 113.2k 107.2k 0.0k 1.1M 1.0M 0.0k nvme5n8 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 131.2k 86.8k 0.0k 1.3M 868.0k 0.0k nvme5n9 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 112.4k 112.0k 0.0k 1.1M 1.1M 0.0k nvme8n1 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 90.8k 106.0k 0.0k 908.0k 1.0M 0.0k nvme8n10 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 105.6k 106.0k 0.0k 1.0M 1.0M 0.0k nvme8n11 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 79.2k 125.2k 0.0k 792.0k 1.2M 0.0k nvme8n12 2024-05-21T12:40:24.888 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 111.6k 87.2k 0.0k 1.1M 872.0k 0.0k nvme8n13 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 99.6k 121.6k 0.0k 996.0k 1.2M 0.0k nvme8n14 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 144.0k 95.6k 0.0k 1.4M 956.0k 0.0k nvme8n15 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 83.6k 100.8k 0.0k 836.0k 1008.0k 0.0k nvme8n16 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 77.6k 137.6k 0.0k 776.0k 1.3M 0.0k nvme8n17 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 96.8k 119.2k 0.0k 968.0k 1.2M 0.0k nvme8n18 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 100.0k 87.6k 0.0k 1000.0k 876.0k 0.0k nvme8n19 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 99.6k 105.6k 0.0k 996.0k 1.0M 0.0k nvme8n2 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 110.4k 72.4k 0.0k 1.1M 724.0k 0.0k nvme8n20 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 115.2k 115.2k 0.0k 1.1M 1.1M 0.0k nvme8n3 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 116.4k 124.4k 0.0k 1.1M 1.2M 0.0k nvme8n4 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 100.0k 110.4k 0.0k 1000.0k 1.1M 0.0k nvme8n5 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 102.8k 107.6k 0.0k 1.0M 1.1M 0.0k nvme8n6 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 97.6k 82.0k 0.0k 976.0k 820.0k 0.0k nvme8n7 2024-05-21T12:40:24.889 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 98.8k 126.4k 0.0k 988.0k 1.2M 0.0k nvme8n8 2024-05-21T12:40:24.890 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 111.2k 128.0k 0.0k 1.1M 1.2M 0.0k nvme8n9 2024-05-21T12:40:24.890 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:26.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:26 smithi045 ceph-mon[30165]: pgmap v828: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 500 op/s 2024-05-21T12:40:26.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:26 smithi072 ceph-mon[33912]: pgmap v828: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 500 op/s 2024-05-21T12:40:26.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:26 smithi092 ceph-mon[33665]: pgmap v828: 33 pgs: 33 active+clean; 1.0 GiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 500 op/s 2024-05-21T12:40:28.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:28 smithi045 ceph-mon[30165]: pgmap v829: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 565 op/s 2024-05-21T12:40:28.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:28 smithi072 ceph-mon[33912]: pgmap v829: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 565 op/s 2024-05-21T12:40:28.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:28 smithi092 ceph-mon[33665]: pgmap v829: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 565 op/s 2024-05-21T12:40:30.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:30 smithi045 ceph-mon[30165]: pgmap v830: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 513 op/s 2024-05-21T12:40:30.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:30 smithi072 ceph-mon[33912]: pgmap v830: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 513 op/s 2024-05-21T12:40:30.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:30 smithi092 ceph-mon[33665]: pgmap v830: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 513 op/s 2024-05-21T12:40:32.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:32 smithi045 ceph-mon[30165]: pgmap v831: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 525 op/s 2024-05-21T12:40:32.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:32 smithi072 ceph-mon[33912]: pgmap v831: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 525 op/s 2024-05-21T12:40:32.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:32 smithi092 ceph-mon[33665]: pgmap v831: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 525 op/s 2024-05-21T12:40:34.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:34 smithi045 ceph-mon[30165]: pgmap v832: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.5 MiB/s wr, 494 op/s 2024-05-21T12:40:34.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:34 smithi072 ceph-mon[33912]: pgmap v832: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.5 MiB/s wr, 494 op/s 2024-05-21T12:40:34.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:34 smithi092 ceph-mon[33665]: pgmap v832: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.5 MiB/s wr, 494 op/s 2024-05-21T12:40:34.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 99.2k 96.0k 0.0k 992.0k 960.0k 0.0k nvme2n1 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.20 98.8k 84.8k 0.0k 988.0k 848.0k 0.0k nvme2n10 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 82.0k 98.0k 0.0k 820.0k 980.0k 0.0k nvme2n11 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 91.6k 70.8k 0.0k 916.0k 708.0k 0.0k nvme2n12 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 62.0k 96.0k 0.0k 620.0k 960.0k 0.0k nvme2n13 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 80.4k 89.2k 0.0k 804.0k 892.0k 0.0k nvme2n14 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 116.4k 93.2k 0.0k 1.1M 932.0k 0.0k nvme2n15 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 77.6k 122.4k 0.0k 776.0k 1.2M 0.0k nvme2n16 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 93.6k 86.8k 0.0k 936.0k 868.0k 0.0k nvme2n17 2024-05-21T12:40:34.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 95.2k 88.0k 0.0k 952.0k 880.0k 0.0k nvme2n18 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 89.2k 78.0k 0.0k 892.0k 780.0k 0.0k nvme2n19 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 57.6k 93.2k 0.0k 576.0k 932.0k 0.0k nvme2n2 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 77.6k 87.2k 0.0k 776.0k 872.0k 0.0k nvme2n20 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 87.6k 103.2k 0.0k 876.0k 1.0M 0.0k nvme2n3 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 72.4k 91.2k 0.0k 724.0k 912.0k 0.0k nvme2n4 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 74.0k 99.2k 0.0k 740.0k 992.0k 0.0k nvme2n5 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 100.4k 82.4k 0.0k 1004.0k 824.0k 0.0k nvme2n6 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 89.2k 66.8k 0.0k 892.0k 668.0k 0.0k nvme2n7 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 76.8k 85.6k 0.0k 768.0k 856.0k 0.0k nvme2n8 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 74.0k 121.2k 0.0k 740.0k 1.2M 0.0k nvme2n9 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 91.2k 93.6k 0.0k 912.0k 936.0k 0.0k nvme5n1 2024-05-21T12:40:34.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.20 81.2k 107.2k 0.0k 812.0k 1.0M 0.0k nvme5n10 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 60.0k 126.4k 0.0k 600.0k 1.2M 0.0k nvme5n11 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.0k 94.8k 0.0k 1.1M 948.0k 0.0k nvme5n12 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 98.4k 103.6k 0.0k 984.0k 1.0M 0.0k nvme5n13 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 120.0k 90.4k 0.0k 1.2M 904.0k 0.0k nvme5n14 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 88.0k 81.6k 0.0k 880.0k 816.0k 0.0k nvme5n15 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 105.6k 72.8k 0.0k 1.0M 728.0k 0.0k nvme5n16 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 88.4k 87.2k 0.0k 884.0k 872.0k 0.0k nvme5n17 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 73.6k 98.4k 0.0k 736.0k 984.0k 0.0k nvme5n18 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 95.6k 82.8k 0.0k 956.0k 828.0k 0.0k nvme5n19 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 82.0k 109.6k 0.0k 820.0k 1.1M 0.0k nvme5n2 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 83.6k 93.2k 0.0k 836.0k 932.0k 0.0k nvme5n20 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 75.2k 124.0k 0.0k 752.0k 1.2M 0.0k nvme5n3 2024-05-21T12:40:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 102.8k 117.2k 0.0k 1.0M 1.1M 0.0k nvme5n4 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 81.6k 76.0k 0.0k 816.0k 760.0k 0.0k nvme5n5 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 99.2k 94.8k 0.0k 992.0k 948.0k 0.0k nvme5n6 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 103.2k 77.6k 0.0k 1.0M 776.0k 0.0k nvme5n7 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 90.8k 82.0k 0.0k 908.0k 820.0k 0.0k nvme5n8 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 67.2k 94.4k 0.0k 672.0k 944.0k 0.0k nvme5n9 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 90.0k 79.2k 0.0k 900.0k 792.0k 0.0k nvme8n1 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.20 94.8k 74.0k 0.0k 948.0k 740.0k 0.0k nvme8n10 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 83.2k 88.0k 0.0k 832.0k 880.0k 0.0k nvme8n11 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 99.6k 88.8k 0.0k 996.0k 888.0k 0.0k nvme8n12 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 109.6k 94.8k 0.0k 1.1M 948.0k 0.0k nvme8n13 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 73.6k 82.0k 0.0k 736.0k 820.0k 0.0k nvme8n14 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 82.4k 92.4k 0.0k 824.0k 924.0k 0.0k nvme8n15 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 128.4k 64.0k 0.0k 1.3M 640.0k 0.0k nvme8n16 2024-05-21T12:40:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 97.6k 91.6k 0.0k 976.0k 916.0k 0.0k nvme8n17 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 78.4k 119.6k 0.0k 784.0k 1.2M 0.0k nvme8n18 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 101.2k 89.2k 0.0k 1012.0k 892.0k 0.0k nvme8n19 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 87.6k 99.6k 0.0k 876.0k 996.0k 0.0k nvme8n2 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 99.2k 94.4k 0.0k 992.0k 944.0k 0.0k nvme8n20 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 115.2k 110.4k 0.0k 1.1M 1.1M 0.0k nvme8n3 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 116.8k 76.4k 0.0k 1.1M 764.0k 0.0k nvme8n4 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 94.0k 99.2k 0.0k 940.0k 992.0k 0.0k nvme8n5 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 94.0k 84.8k 0.0k 940.0k 848.0k 0.0k nvme8n6 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 98.8k 72.4k 0.0k 988.0k 724.0k 0.0k nvme8n7 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 98.0k 75.6k 0.0k 980.0k 756.0k 0.0k nvme8n8 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 91.6k 96.8k 0.0k 916.0k 968.0k 0.0k nvme8n9 2024-05-21T12:40:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:36.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:36 smithi045 ceph-mon[30165]: pgmap v833: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 504 op/s 2024-05-21T12:40:36.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:36 smithi072 ceph-mon[33912]: pgmap v833: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 504 op/s 2024-05-21T12:40:36.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:36 smithi092 ceph-mon[33665]: pgmap v833: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 504 op/s 2024-05-21T12:40:38.318 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:38 smithi045 ceph-mon[30165]: pgmap v834: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-05-21T12:40:38.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:38 smithi072 ceph-mon[33912]: pgmap v834: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-05-21T12:40:38.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:38 smithi092 ceph-mon[33665]: pgmap v834: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 557 op/s 2024-05-21T12:40:40.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:40 smithi045 ceph-mon[30165]: pgmap v835: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 505 op/s 2024-05-21T12:40:40.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:40 smithi072 ceph-mon[33912]: pgmap v835: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 505 op/s 2024-05-21T12:40:40.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:40 smithi092 ceph-mon[33665]: pgmap v835: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 505 op/s 2024-05-21T12:40:42.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:42 smithi045 ceph-mon[30165]: pgmap v836: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-21T12:40:42.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:42 smithi072 ceph-mon[33912]: pgmap v836: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-21T12:40:42.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:42 smithi092 ceph-mon[33665]: pgmap v836: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 519 op/s 2024-05-21T12:40:44.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:44 smithi045 ceph-mon[30165]: pgmap v837: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-05-21T12:40:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:44 smithi072 ceph-mon[33912]: pgmap v837: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-05-21T12:40:44.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:44 smithi092 ceph-mon[33665]: pgmap v837: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 487 op/s 2024-05-21T12:40:44.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 99.2k 72.4k 0.0k 992.0k 724.0k 0.0k nvme2n1 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 75.2k 100.0k 0.0k 752.0k 1000.0k 0.0k nvme2n10 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 78.8k 95.2k 0.0k 788.0k 952.0k 0.0k nvme2n11 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 74.4k 96.8k 0.0k 744.0k 968.0k 0.0k nvme2n12 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 68.4k 71.6k 0.0k 684.0k 716.0k 0.0k nvme2n13 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 115.2k 66.8k 0.0k 1.1M 668.0k 0.0k nvme2n14 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 95.2k 80.4k 0.0k 952.0k 804.0k 0.0k nvme2n15 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 99.6k 72.8k 0.0k 996.0k 728.0k 0.0k nvme2n16 2024-05-21T12:40:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 70.0k 109.6k 0.0k 700.0k 1.1M 0.0k nvme2n17 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 83.6k 78.0k 0.0k 836.0k 780.0k 0.0k nvme2n18 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 100.0k 100.0k 0.0k 1000.0k 1000.0k 0.0k nvme2n19 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 76.0k 94.8k 0.0k 760.0k 948.0k 0.0k nvme2n2 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 106.8k 67.6k 0.0k 1.0M 676.0k 0.0k nvme2n20 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 94.4k 76.8k 0.0k 944.0k 768.0k 0.0k nvme2n3 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 92.8k 90.4k 0.0k 928.0k 904.0k 0.0k nvme2n4 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 70.4k 104.8k 0.0k 704.0k 1.0M 0.0k nvme2n5 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 104.0k 64.0k 0.0k 1.0M 640.0k 0.0k nvme2n6 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 89.2k 79.6k 0.0k 892.0k 796.0k 0.0k nvme2n7 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 83.2k 87.2k 0.0k 832.0k 872.0k 0.0k nvme2n8 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 90.8k 81.6k 0.0k 908.0k 816.0k 0.0k nvme2n9 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 63.6k 103.2k 0.0k 636.0k 1.0M 0.0k nvme5n1 2024-05-21T12:40:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 92.8k 83.2k 0.0k 928.0k 832.0k 0.0k nvme5n10 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 80.4k 87.2k 0.0k 804.0k 872.0k 0.0k nvme5n11 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 79.6k 98.4k 0.0k 796.0k 984.0k 0.0k nvme5n12 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 106.8k 101.2k 0.0k 1.0M 1012.0k 0.0k nvme5n13 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 83.6k 79.6k 0.0k 836.0k 796.0k 0.0k nvme5n14 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 88.4k 85.2k 0.0k 884.0k 852.0k 0.0k nvme5n15 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 95.6k 84.8k 0.0k 956.0k 848.0k 0.0k nvme5n16 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 84.4k 83.2k 0.0k 844.0k 832.0k 0.0k nvme5n17 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 106.4k 72.8k 0.0k 1.0M 728.0k 0.0k nvme5n18 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 86.0k 60.8k 0.0k 860.0k 608.0k 0.0k nvme5n19 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 71.2k 82.0k 0.0k 712.0k 820.0k 0.0k nvme5n2 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 84.0k 92.4k 0.0k 840.0k 924.0k 0.0k nvme5n20 2024-05-21T12:40:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 90.8k 76.4k 0.0k 908.0k 764.0k 0.0k nvme5n3 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 99.2k 94.4k 0.0k 992.0k 944.0k 0.0k nvme5n4 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 101.6k 75.2k 0.0k 1016.0k 752.0k 0.0k nvme5n5 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 78.4k 97.2k 0.0k 784.0k 972.0k 0.0k nvme5n6 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 74.4k 66.0k 0.0k 744.0k 660.0k 0.0k nvme5n7 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 100.0k 94.0k 0.0k 1000.0k 940.0k 0.0k nvme5n8 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 83.2k 96.0k 0.0k 832.0k 960.0k 0.0k nvme5n9 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 107.6k 72.4k 0.0k 1.1M 724.0k 0.0k nvme8n1 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.60 70.8k 94.4k 0.0k 708.0k 944.0k 0.0k nvme8n10 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 82.0k 94.8k 0.0k 820.0k 948.0k 0.0k nvme8n11 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 99.2k 84.4k 0.0k 992.0k 844.0k 0.0k nvme8n12 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 102.8k 98.0k 0.0k 1.0M 980.0k 0.0k nvme8n13 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 79.6k 98.8k 0.0k 796.0k 988.0k 0.0k nvme8n14 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 130.8k 76.0k 0.0k 1.3M 760.0k 0.0k nvme8n15 2024-05-21T12:40:44.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 97.6k 73.2k 0.0k 976.0k 732.0k 0.0k nvme8n16 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 98.8k 86.0k 0.0k 988.0k 860.0k 0.0k nvme8n17 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 94.8k 68.8k 0.0k 948.0k 688.0k 0.0k nvme8n18 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 96.8k 72.4k 0.0k 968.0k 724.0k 0.0k nvme8n19 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 90.8k 90.0k 0.0k 908.0k 900.0k 0.0k nvme8n2 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 101.6k 77.6k 0.0k 1016.0k 776.0k 0.0k nvme8n20 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 107.6k 90.0k 0.0k 1.1M 900.0k 0.0k nvme8n3 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 92.0k 101.2k 0.0k 920.0k 1012.0k 0.0k nvme8n4 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 103.2k 82.0k 0.0k 1.0M 820.0k 0.0k nvme8n5 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 66.0k 108.4k 0.0k 660.0k 1.1M 0.0k nvme8n6 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 108.0k 66.8k 0.0k 1.1M 668.0k 0.0k nvme8n7 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 98.8k 64.4k 0.0k 988.0k 644.0k 0.0k nvme8n8 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.50 74.8k 94.0k 0.0k 748.0k 940.0k 0.0k nvme8n9 2024-05-21T12:40:44.884 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:46.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:46 smithi045 ceph-mon[30165]: pgmap v838: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 494 op/s 2024-05-21T12:40:46.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:46 smithi072 ceph-mon[33912]: pgmap v838: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 494 op/s 2024-05-21T12:40:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:46 smithi092 ceph-mon[33665]: pgmap v838: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 494 op/s 2024-05-21T12:40:48.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:48 smithi045 ceph-mon[30165]: pgmap v839: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 556 op/s 2024-05-21T12:40:48.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:48 smithi072 ceph-mon[33912]: pgmap v839: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 556 op/s 2024-05-21T12:40:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:48 smithi092 ceph-mon[33665]: pgmap v839: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 556 op/s 2024-05-21T12:40:50.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:50 smithi045 ceph-mon[30165]: pgmap v840: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 504 op/s 2024-05-21T12:40:50.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:50 smithi072 ceph-mon[33912]: pgmap v840: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 504 op/s 2024-05-21T12:40:50.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:50 smithi092 ceph-mon[33665]: pgmap v840: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 504 op/s 2024-05-21T12:40:52.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:52 smithi045 ceph-mon[30165]: pgmap v841: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-21T12:40:52.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:52 smithi072 ceph-mon[33912]: pgmap v841: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-21T12:40:52.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:52 smithi092 ceph-mon[33665]: pgmap v841: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-21T12:40:54.304 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:54 smithi092 ceph-mon[33665]: pgmap v842: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 485 op/s 2024-05-21T12:40:54.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:54 smithi045 ceph-mon[30165]: pgmap v842: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 485 op/s 2024-05-21T12:40:54.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:54 smithi072 ceph-mon[33912]: pgmap v842: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 485 op/s 2024-05-21T12:40:54.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:54.890 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:40:54.890 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 80.8k 78.4k 0.0k 808.0k 784.0k 0.0k nvme2n1 2024-05-21T12:40:54.890 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 56.0k 72.8k 0.0k 560.0k 728.0k 0.0k nvme2n10 2024-05-21T12:40:54.890 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 86.8k 96.4k 0.0k 868.0k 964.0k 0.0k nvme2n11 2024-05-21T12:40:54.890 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 80.8k 92.4k 0.0k 808.0k 924.0k 0.0k nvme2n12 2024-05-21T12:40:54.890 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 71.6k 93.2k 0.0k 716.0k 932.0k 0.0k nvme2n13 2024-05-21T12:40:54.890 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 52.4k 106.8k 0.0k 524.0k 1.0M 0.0k nvme2n14 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 73.6k 81.2k 0.0k 736.0k 812.0k 0.0k nvme2n15 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 79.2k 84.4k 0.0k 792.0k 844.0k 0.0k nvme2n16 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 103.2k 84.8k 0.0k 1.0M 848.0k 0.0k nvme2n17 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 74.4k 83.6k 0.0k 744.0k 836.0k 0.0k nvme2n18 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 76.8k 78.4k 0.0k 768.0k 784.0k 0.0k nvme2n19 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 78.0k 88.0k 0.0k 780.0k 880.0k 0.0k nvme2n2 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 101.6k 97.6k 0.0k 1016.0k 976.0k 0.0k nvme2n20 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 84.4k 74.4k 0.0k 844.0k 744.0k 0.0k nvme2n3 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 77.2k 71.2k 0.0k 772.0k 712.0k 0.0k nvme2n4 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 81.2k 82.4k 0.0k 812.0k 824.0k 0.0k nvme2n5 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 74.8k 69.6k 0.0k 748.0k 696.0k 0.0k nvme2n6 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 89.6k 131.2k 0.0k 896.0k 1.3M 0.0k nvme2n7 2024-05-21T12:40:54.891 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 78.0k 84.4k 0.0k 780.0k 844.0k 0.0k nvme2n8 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 60.0k 97.2k 0.0k 600.0k 972.0k 0.0k nvme2n9 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 96.8k 80.4k 0.0k 968.0k 804.0k 0.0k nvme5n1 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 43.2k 92.8k 0.0k 432.0k 928.0k 0.0k nvme5n10 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 65.2k 80.8k 0.0k 652.0k 808.0k 0.0k nvme5n11 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 64.8k 95.2k 0.0k 648.0k 952.0k 0.0k nvme5n12 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 98.0k 69.2k 0.0k 980.0k 692.0k 0.0k nvme5n13 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 88.8k 92.4k 0.0k 888.0k 924.0k 0.0k nvme5n14 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 74.8k 90.4k 0.0k 748.0k 904.0k 0.0k nvme5n15 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 84.8k 83.6k 0.0k 848.0k 836.0k 0.0k nvme5n16 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 58.0k 85.6k 0.0k 580.0k 856.0k 0.0k nvme5n17 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 63.6k 88.8k 0.0k 636.0k 888.0k 0.0k nvme5n18 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 98.4k 90.8k 0.0k 984.0k 908.0k 0.0k nvme5n19 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 82.4k 74.4k 0.0k 824.0k 744.0k 0.0k nvme5n2 2024-05-21T12:40:54.892 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 84.4k 91.2k 0.0k 844.0k 912.0k 0.0k nvme5n20 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 63.6k 102.8k 0.0k 636.0k 1.0M 0.0k nvme5n3 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 79.6k 81.2k 0.0k 796.0k 812.0k 0.0k nvme5n4 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 72.4k 74.0k 0.0k 724.0k 740.0k 0.0k nvme5n5 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 65.2k 88.4k 0.0k 652.0k 884.0k 0.0k nvme5n6 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 84.8k 70.4k 0.0k 848.0k 704.0k 0.0k nvme5n7 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 70.0k 82.8k 0.0k 700.0k 828.0k 0.0k nvme5n8 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 64.4k 84.0k 0.0k 644.0k 840.0k 0.0k nvme5n9 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 81.6k 98.0k 0.0k 816.0k 980.0k 0.0k nvme8n1 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 92.8k 70.4k 0.0k 928.0k 704.0k 0.0k nvme8n10 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 71.2k 92.0k 0.0k 712.0k 920.0k 0.0k nvme8n11 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 89.6k 75.6k 0.0k 896.0k 756.0k 0.0k nvme8n12 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 98.4k 77.6k 0.0k 984.0k 776.0k 0.0k nvme8n13 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 80.8k 66.0k 0.0k 808.0k 660.0k 0.0k nvme8n14 2024-05-21T12:40:54.893 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 61.2k 118.8k 0.0k 612.0k 1.2M 0.0k nvme8n15 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.40 87.2k 66.4k 0.0k 872.0k 664.0k 0.0k nvme8n16 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 68.4k 75.2k 0.0k 684.0k 752.0k 0.0k nvme8n17 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 68.8k 86.0k 0.0k 688.0k 860.0k 0.0k nvme8n18 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 72.4k 93.2k 0.0k 724.0k 932.0k 0.0k nvme8n19 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 78.8k 96.8k 0.0k 788.0k 968.0k 0.0k nvme8n2 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 68.4k 76.4k 0.0k 684.0k 764.0k 0.0k nvme8n20 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 80.4k 91.6k 0.0k 804.0k 916.0k 0.0k nvme8n3 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 80.0k 91.2k 0.0k 800.0k 912.0k 0.0k nvme8n4 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 84.0k 70.4k 0.0k 840.0k 704.0k 0.0k nvme8n5 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 105.2k 74.8k 0.0k 1.0M 748.0k 0.0k nvme8n6 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 60.4k 93.6k 0.0k 604.0k 936.0k 0.0k nvme8n7 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 62.8k 87.6k 0.0k 628.0k 876.0k 0.0k nvme8n8 2024-05-21T12:40:54.894 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 88.8k 100.0k 0.0k 888.0k 1000.0k 0.0k nvme8n9 2024-05-21T12:40:54.895 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:40:56.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:56 smithi045 ceph-mon[30165]: pgmap v843: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 493 op/s 2024-05-21T12:40:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:56 smithi072 ceph-mon[33912]: pgmap v843: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 493 op/s 2024-05-21T12:40:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:56 smithi092 ceph-mon[33665]: pgmap v843: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 493 op/s 2024-05-21T12:40:58.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:40:58 smithi045 ceph-mon[30165]: pgmap v844: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 5.4 MiB/s wr, 550 op/s 2024-05-21T12:40:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:40:58 smithi072 ceph-mon[33912]: pgmap v844: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 5.4 MiB/s wr, 550 op/s 2024-05-21T12:40:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:40:58 smithi092 ceph-mon[33665]: pgmap v844: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.2 MiB/s rd, 5.4 MiB/s wr, 550 op/s 2024-05-21T12:41:00.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:00 smithi045 ceph-mon[30165]: pgmap v845: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-21T12:41:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:00 smithi072 ceph-mon[33912]: pgmap v845: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-21T12:41:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:00 smithi092 ceph-mon[33665]: pgmap v845: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-21T12:41:01.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:01 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:41:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:01 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:41:01.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:01 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:41:02.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:41:02.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:41:02.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:02 smithi045 ceph-mon[30165]: pgmap v846: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 472 op/s 2024-05-21T12:41:02.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:02 smithi045 ceph-mon[30165]: pgmap v847: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 456 op/s 2024-05-21T12:41:02.316 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:41:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:02 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:41:02.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:02 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:41:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:02 smithi072 ceph-mon[33912]: pgmap v846: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 472 op/s 2024-05-21T12:41:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:02 smithi072 ceph-mon[33912]: pgmap v847: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 456 op/s 2024-05-21T12:41:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:02 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:41:02.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:02 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:41:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:02 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:41:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:02 smithi092 ceph-mon[33665]: pgmap v846: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 472 op/s 2024-05-21T12:41:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:02 smithi092 ceph-mon[33665]: pgmap v847: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 456 op/s 2024-05-21T12:41:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:02 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:41:04.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:04 smithi072 ceph-mon[33912]: pgmap v848: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 538 op/s 2024-05-21T12:41:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:04 smithi092 ceph-mon[33665]: pgmap v848: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 538 op/s 2024-05-21T12:41:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:04 smithi045 ceph-mon[30165]: pgmap v848: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 538 op/s 2024-05-21T12:41:04.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:04.880 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 133.2k 114.4k 0.0k 1.3M 1.1M 0.0k nvme2n1 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 124.0k 64.8k 0.0k 1.2M 648.0k 0.0k nvme2n10 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 88.4k 85.2k 0.0k 884.0k 852.0k 0.0k nvme2n11 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 74.4k 82.0k 0.0k 744.0k 820.0k 0.0k nvme2n12 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 97.2k 84.4k 0.0k 972.0k 844.0k 0.0k nvme2n13 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 83.2k 72.0k 0.0k 832.0k 720.0k 0.0k nvme2n14 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 120.0k 110.8k 0.0k 1.2M 1.1M 0.0k nvme2n15 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 76.4k 71.6k 0.0k 764.0k 716.0k 0.0k nvme2n16 2024-05-21T12:41:04.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 106.0k 62.4k 0.0k 1.0M 624.0k 0.0k nvme2n17 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.4k 127.2k 0.0k 1.1M 1.2M 0.0k nvme2n18 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 81.2k 71.6k 0.0k 812.0k 716.0k 0.0k nvme2n19 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 81.2k 90.8k 0.0k 812.0k 908.0k 0.0k nvme2n2 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 101.6k 159.2k 0.0k 1016.0k 1.6M 0.0k nvme2n20 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 75.2k 65.6k 0.0k 752.0k 656.0k 0.0k nvme2n3 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 117.2k 82.0k 0.0k 1.1M 820.0k 0.0k nvme2n4 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 74.0k 66.0k 0.0k 740.0k 660.0k 0.0k nvme2n5 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 140.8k 123.2k 0.0k 1.4M 1.2M 0.0k nvme2n6 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 162.4k 96.4k 0.0k 1.6M 964.0k 0.0k nvme2n7 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 79.2k 55.6k 0.0k 792.0k 556.0k 0.0k nvme2n8 2024-05-21T12:41:04.882 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 104.8k 108.4k 0.0k 1.0M 1.1M 0.0k nvme2n9 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 156.8k 129.2k 0.0k 1.5M 1.3M 0.0k nvme5n1 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 71.2k 82.8k 0.0k 712.0k 828.0k 0.0k nvme5n10 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 95.2k 86.8k 0.0k 952.0k 868.0k 0.0k nvme5n11 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 89.6k 70.8k 0.0k 896.0k 708.0k 0.0k nvme5n12 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 82.4k 73.2k 0.0k 824.0k 732.0k 0.0k nvme5n13 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 78.0k 85.6k 0.0k 780.0k 856.0k 0.0k nvme5n14 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 83.6k 65.2k 0.0k 836.0k 652.0k 0.0k nvme5n15 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 94.8k 118.0k 0.0k 948.0k 1.2M 0.0k nvme5n16 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 76.4k 141.2k 0.0k 764.0k 1.4M 0.0k nvme5n17 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 79.6k 84.0k 0.0k 796.0k 840.0k 0.0k nvme5n18 2024-05-21T12:41:04.883 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 59.6k 91.6k 0.0k 596.0k 916.0k 0.0k nvme5n19 2024-05-21T12:41:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 99.6k 93.2k 0.0k 996.0k 932.0k 0.0k nvme5n2 2024-05-21T12:41:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 74.0k 94.0k 0.0k 740.0k 940.0k 0.0k nvme5n20 2024-05-21T12:41:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 77.6k 63.2k 0.0k 776.0k 632.0k 0.0k nvme5n3 2024-05-21T12:41:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 86.0k 84.8k 0.0k 860.0k 848.0k 0.0k nvme5n4 2024-05-21T12:41:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 85.2k 78.4k 0.0k 852.0k 784.0k 0.0k nvme5n5 2024-05-21T12:41:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 70.8k 87.2k 0.0k 708.0k 872.0k 0.0k nvme5n6 2024-05-21T12:41:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 126.0k 100.4k 0.0k 1.2M 1004.0k 0.0k nvme5n7 2024-05-21T12:41:04.884 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 64.0k 72.8k 0.0k 640.0k 728.0k 0.0k nvme5n8 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 57.2k 94.8k 0.0k 572.0k 948.0k 0.0k nvme5n9 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 72.8k 80.8k 0.0k 728.0k 808.0k 0.0k nvme8n1 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 83.6k 112.8k 0.0k 836.0k 1.1M 0.0k nvme8n10 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 95.6k 84.4k 0.0k 956.0k 844.0k 0.0k nvme8n11 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 102.0k 140.4k 0.0k 1020.0k 1.4M 0.0k nvme8n12 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 63.2k 90.4k 0.0k 632.0k 904.0k 0.0k nvme8n13 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 154.0k 143.6k 0.0k 1.5M 1.4M 0.0k nvme8n14 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 83.6k 87.2k 0.0k 836.0k 872.0k 0.0k nvme8n15 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 159.6k 114.0k 0.0k 1.6M 1.1M 0.0k nvme8n16 2024-05-21T12:41:04.885 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 56.4k 93.2k 0.0k 564.0k 932.0k 0.0k nvme8n17 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 72.0k 93.2k 0.0k 720.0k 932.0k 0.0k nvme8n18 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 108.4k 51.2k 0.0k 1.1M 512.0k 0.0k nvme8n19 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 82.0k 84.8k 0.0k 820.0k 848.0k 0.0k nvme8n2 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 66.4k 74.4k 0.0k 664.0k 744.0k 0.0k nvme8n20 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 72.0k 90.4k 0.0k 720.0k 904.0k 0.0k nvme8n3 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.8k 68.4k 0.0k 1.1M 684.0k 0.0k nvme8n4 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 90.8k 76.0k 0.0k 908.0k 760.0k 0.0k nvme8n5 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 110.0k 110.4k 0.0k 1.1M 1.1M 0.0k nvme8n6 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 104.8k 80.0k 0.0k 1.0M 800.0k 0.0k nvme8n7 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 112.0k 88.8k 0.0k 1.1M 888.0k 0.0k nvme8n8 2024-05-21T12:41:04.886 INFO:tasks.workunit.client.3.smithi137.stdout: 8.30 67.6k 78.8k 0.0k 676.0k 788.0k 0.0k nvme8n9 2024-05-21T12:41:04.887 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:06.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:06 smithi072 ceph-mon[33912]: pgmap v849: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-05-21T12:41:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:06 smithi092 ceph-mon[33665]: pgmap v849: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-05-21T12:41:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:06 smithi045 ceph-mon[30165]: pgmap v849: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-05-21T12:41:08.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:08 smithi072 ceph-mon[33912]: pgmap v850: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 497 op/s 2024-05-21T12:41:08.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:08 smithi092 ceph-mon[33665]: pgmap v850: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 497 op/s 2024-05-21T12:41:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:08 smithi045 ceph-mon[30165]: pgmap v850: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 497 op/s 2024-05-21T12:41:10.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:10 smithi072 ceph-mon[33912]: pgmap v851: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 464 op/s 2024-05-21T12:41:10.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:10 smithi092 ceph-mon[33665]: pgmap v851: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 464 op/s 2024-05-21T12:41:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:10 smithi045 ceph-mon[30165]: pgmap v851: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 464 op/s 2024-05-21T12:41:12.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:12 smithi072 ceph-mon[33912]: pgmap v852: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 487 op/s 2024-05-21T12:41:12.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:12 smithi092 ceph-mon[33665]: pgmap v852: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 487 op/s 2024-05-21T12:41:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:12 smithi045 ceph-mon[30165]: pgmap v852: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 487 op/s 2024-05-21T12:41:14.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:14 smithi072 ceph-mon[33912]: pgmap v853: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 523 op/s 2024-05-21T12:41:14.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:14 smithi092 ceph-mon[33665]: pgmap v853: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 523 op/s 2024-05-21T12:41:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:14 smithi045 ceph-mon[30165]: pgmap v853: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 523 op/s 2024-05-21T12:41:14.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:41:14.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 105.6k 143.6k 0.0k 1.0M 1.4M 0.0k nvme2n1 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 134.0k 126.8k 0.0k 1.3M 1.2M 0.0k nvme2n10 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 50.0k 92.8k 0.0k 500.0k 928.0k 0.0k nvme2n11 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 77.2k 72.4k 0.0k 772.0k 724.0k 0.0k nvme2n12 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 108.4k 137.2k 0.0k 1.1M 1.3M 0.0k nvme2n13 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 114.8k 103.2k 0.0k 1.1M 1.0M 0.0k nvme2n14 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 139.6k 118.8k 0.0k 1.4M 1.2M 0.0k nvme2n15 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 96.4k 100.0k 0.0k 964.0k 1000.0k 0.0k nvme2n16 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 141.6k 130.4k 0.0k 1.4M 1.3M 0.0k nvme2n17 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 146.0k 98.8k 0.0k 1.4M 988.0k 0.0k nvme2n18 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 112.8k 118.4k 0.0k 1.1M 1.2M 0.0k nvme2n19 2024-05-21T12:41:14.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 133.2k 134.0k 0.0k 1.3M 1.3M 0.0k nvme2n2 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 106.8k 108.0k 0.0k 1.0M 1.1M 0.0k nvme2n20 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 170.8k 111.2k 0.0k 1.7M 1.1M 0.0k nvme2n3 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 102.4k 114.4k 0.0k 1.0M 1.1M 0.0k nvme2n4 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 78.8k 101.2k 0.0k 788.0k 1012.0k 0.0k nvme2n5 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 105.6k 83.6k 0.0k 1.0M 836.0k 0.0k nvme2n6 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 102.8k 99.2k 0.0k 1.0M 992.0k 0.0k nvme2n7 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 116.0k 129.6k 0.0k 1.1M 1.3M 0.0k nvme2n8 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 126.4k 112.0k 0.0k 1.2M 1.1M 0.0k nvme2n9 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 124.0k 104.8k 0.0k 1.2M 1.0M 0.0k nvme5n1 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 61.2k 70.0k 0.0k 612.0k 700.0k 0.0k nvme5n10 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 126.8k 112.4k 0.0k 1.2M 1.1M 0.0k nvme5n11 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 105.6k 84.4k 0.0k 1.0M 844.0k 0.0k nvme5n12 2024-05-21T12:41:14.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 125.6k 135.2k 0.0k 1.2M 1.3M 0.0k nvme5n13 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 95.2k 84.4k 0.0k 952.0k 844.0k 0.0k nvme5n14 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 67.2k 96.4k 0.0k 672.0k 964.0k 0.0k nvme5n15 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 112.4k 142.8k 0.0k 1.1M 1.4M 0.0k nvme5n16 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 113.6k 148.0k 0.0k 1.1M 1.4M 0.0k nvme5n17 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 108.0k 103.2k 0.0k 1.1M 1.0M 0.0k nvme5n18 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 120.0k 94.0k 0.0k 1.2M 940.0k 0.0k nvme5n19 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 116.4k 137.6k 0.0k 1.1M 1.3M 0.0k nvme5n2 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 147.6k 122.8k 0.0k 1.4M 1.2M 0.0k nvme5n20 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 95.2k 128.4k 0.0k 952.0k 1.3M 0.0k nvme5n3 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 54.8k 55.6k 0.0k 548.0k 556.0k 0.0k nvme5n4 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 122.8k 128.8k 0.0k 1.2M 1.3M 0.0k nvme5n5 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 58.4k 104.0k 0.0k 584.0k 1.0M 0.0k nvme5n6 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 111.6k 100.0k 0.0k 1.1M 1000.0k 0.0k nvme5n7 2024-05-21T12:41:14.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 91.6k 92.8k 0.0k 916.0k 928.0k 0.0k nvme5n8 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 109.6k 114.4k 0.0k 1.1M 1.1M 0.0k nvme5n9 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 112.8k 116.4k 0.0k 1.1M 1.1M 0.0k nvme8n1 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 120.0k 124.4k 0.0k 1.2M 1.2M 0.0k nvme8n10 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 116.4k 118.4k 0.0k 1.1M 1.2M 0.0k nvme8n11 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 119.2k 121.2k 0.0k 1.2M 1.2M 0.0k nvme8n12 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 68.8k 60.8k 0.0k 688.0k 608.0k 0.0k nvme8n13 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 108.8k 138.0k 0.0k 1.1M 1.3M 0.0k nvme8n14 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 76.0k 74.4k 0.0k 760.0k 744.0k 0.0k nvme8n15 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 83.2k 114.8k 0.0k 832.0k 1.1M 0.0k nvme8n16 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 123.6k 101.2k 0.0k 1.2M 1012.0k 0.0k nvme8n17 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 116.0k 138.4k 0.0k 1.1M 1.4M 0.0k nvme8n18 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 146.4k 96.0k 0.0k 1.4M 960.0k 0.0k nvme8n19 2024-05-21T12:41:14.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 100.8k 143.6k 0.0k 1008.0k 1.4M 0.0k nvme8n2 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 106.8k 134.8k 0.0k 1.0M 1.3M 0.0k nvme8n20 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 90.8k 67.2k 0.0k 908.0k 672.0k 0.0k nvme8n3 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 101.2k 148.0k 0.0k 1012.0k 1.4M 0.0k nvme8n4 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 104.8k 133.6k 0.0k 1.0M 1.3M 0.0k nvme8n5 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 106.8k 125.2k 0.0k 1.0M 1.2M 0.0k nvme8n6 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 111.2k 141.2k 0.0k 1.1M 1.4M 0.0k nvme8n7 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 130.4k 116.8k 0.0k 1.3M 1.1M 0.0k nvme8n8 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.80 84.0k 58.8k 0.0k 840.0k 588.0k 0.0k nvme8n9 2024-05-21T12:41:14.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:16.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:16 smithi072 ceph-mon[33912]: pgmap v854: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 477 op/s 2024-05-21T12:41:16.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:16 smithi092 ceph-mon[33665]: pgmap v854: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 477 op/s 2024-05-21T12:41:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:16 smithi045 ceph-mon[30165]: pgmap v854: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 477 op/s 2024-05-21T12:41:18.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:18 smithi072 ceph-mon[33912]: pgmap v855: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.5 MiB/s wr, 486 op/s 2024-05-21T12:41:18.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:18 smithi092 ceph-mon[33665]: pgmap v855: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.5 MiB/s wr, 486 op/s 2024-05-21T12:41:18.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:18 smithi045 ceph-mon[30165]: pgmap v855: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.3 MiB/s rd, 6.5 MiB/s wr, 486 op/s 2024-05-21T12:41:20.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:20 smithi072 ceph-mon[33912]: pgmap v856: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 455 op/s 2024-05-21T12:41:20.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:20 smithi092 ceph-mon[33665]: pgmap v856: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 455 op/s 2024-05-21T12:41:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:20 smithi045 ceph-mon[30165]: pgmap v856: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 455 op/s 2024-05-21T12:41:21.339 INFO:tasks.nvmeof.[nvmeof.thrasher]:done waiting before thrashing 2024-05-21T12:41:21.340 INFO:tasks.nvmeof.[nvmeof.thrasher]:display and verify stats: 2024-05-21T12:41:21.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:41:21.376 INFO:teuthology.orchestra.run.smithi045.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service - Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:41:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:41:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: Active: active (running) since Tue 2024-05-21 12:39:51 UTC; 1min 29s ago 2024-05-21T12:41:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76953 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:41:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76954 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:41:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: Main PID: 77017 (conmon) 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: Tasks: 126 (limit: 202854) 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: Memory: 147.5M 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: CPU: 1min 31.698s 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: ├─libpod-payload-40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77019 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77021 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77039 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.a --gateway-address 172.21.15.45:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.45:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.a -k /etc/ceph/keyring 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77090 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: │ └─77886 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.377 INFO:teuthology.orchestra.run.smithi045.stdout: └─runtime 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: └─77017 /usr/bin/conmon --api-version 1 -c 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f -u 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata -p /run/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 1 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 1 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: ], , client address: IPv4 172.21.15.45:51938 2024-05-21T12:41:21.378 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:39:57 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:39:57.966555] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_1_0 elapsed time: 0.000228 seconds 2024-05-21T12:41:21.379 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service - Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: Active: active (running) since Tue 2024-05-21 12:36:12 UTC; 5min ago 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56908 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56909 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: Main PID: 56971 (conmon) 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: Tasks: 126 (limit: 202814) 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: Memory: 155.5M 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: CPU: 5min 18.006s 2024-05-21T12:41:21.412 INFO:teuthology.orchestra.run.smithi072.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: ├─libpod-payload-8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56973 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56975 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56993 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.b --gateway-address 172.21.15.72:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.72:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.b -k /etc/ceph/keyring 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─57044 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: │ └─57834 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: └─runtime 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: └─56971 /usr/bin/conmon --api-version 1 -c 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -u 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata -p /run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:41:21.413 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:41:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:41:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:41:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:41:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:41:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:41:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:41:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:51082 2024-05-21T12:41:21.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:41:21.451 INFO:teuthology.orchestra.run.smithi092.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service - Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: Active: active (running) since Tue 2024-05-21 12:39:52 UTC; 1min 29s ago 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 58193 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 58194 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: Main PID: 58253 (conmon) 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: Tasks: 126 (limit: 202816) 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: Memory: 141.7M 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: CPU: 1min 30.984s 2024-05-21T12:41:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: ├─libpod-payload-61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─58255 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─58257 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─58274 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.c --gateway-address 172.21.15.92:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.92:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.c -k /etc/ceph/keyring 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─58325 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: │ └─59128 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: └─runtime 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: └─58253 /usr/bin/conmon --api-version 1 -c 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 -u 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615/userdata -p /run/containers/storage/overlay-containers/61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 3 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: OPTIMIZED 2024-05-21T12:41:21.453 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:41:21.454 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:41:21.454 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: states { 2024-05-21T12:41:21.454 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: grp_id: 3 2024-05-21T12:41:21.454 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: state: OPTIMIZED 2024-05-21T12:41:21.454 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: } 2024-05-21T12:41:21.454 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: ], , client address: IPv4 172.21.15.92:37124 2024-05-21T12:41:21.454 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:39:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [2024-05-21 12:39:57.967197] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000285 seconds 2024-05-21T12:41:21.455 DEBUG:teuthology.orchestra.run.smithi137:> ceph orch ls && ceph orch ps --daemon-type nvmeof && ceph health detail && ceph -s && sudo nvme list-subsys /dev/nvme5n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n20 | grep 'live optimized' 2024-05-21T12:41:21.764 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T12:41:21.764 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 1/1 6m ago 22m smithi045=x;count:1 2024-05-21T12:41:21.764 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 6m ago 23m smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:41:21.764 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.mypool ?:4420,5500,8009 1/3 6m ago 17m smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:41:21.764 INFO:teuthology.orchestra.run.smithi137.stdout:osd 8 6m ago - 2024-05-21T12:41:22.065 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:41:22.065 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.a smithi045 *:5500,4420,8009 error 6m ago 16m - - 2024-05-21T12:41:22.065 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.b smithi072 *:5500,4420,8009 error 6m ago 17m - - 2024-05-21T12:41:22.066 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.c smithi092 *:5500,4420,8009 running (17m) 6m ago 17m 160M - 1.2.9 30dd525ace30 5e5a8ffd2ba6 2024-05-21T12:41:22.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:22 smithi072 ceph-mon[33912]: pgmap v857: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 460 op/s 2024-05-21T12:41:22.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:22 smithi072 ceph-mon[33912]: from='client.25294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:41:22.420 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:41:22.420 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:41:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:41:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:41:22.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:22 smithi092 ceph-mon[33665]: pgmap v857: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 460 op/s 2024-05-21T12:41:22.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:22 smithi092 ceph-mon[33665]: from='client.25294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:41:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:22 smithi045 ceph-mon[30165]: pgmap v857: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 460 op/s 2024-05-21T12:41:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:22 smithi045 ceph-mon[30165]: from='client.25294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:41:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: cluster: 2024-05-21T12:41:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:41:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: health: HEALTH_WARN 2024-05-21T12:41:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: 2 failed cephadm daemon(s) 2024-05-21T12:41:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:41:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: services: 2024-05-21T12:41:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: mon: 3 daemons, quorum a,c,b (age 22m) 2024-05-21T12:41:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: mgr: x(active, since 23m) 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: osd: 8 osds: 8 up (since 19m), 8 in (since 19m) 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: data: 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: pools: 2 pools, 33 pgs 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: objects: 485 objects, 1.0 GiB 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: usage: 4.2 GiB used, 711 GiB / 715 GiB avail 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: pgs: 33 active+clean 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: io: 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: client: 6.2 MiB/s rd, 6.4 MiB/s wr, 226 op/s rd, 233 op/s wr 2024-05-21T12:41:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:41:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.841 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.860 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.953 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.064 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.156 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.174 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.193 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.229 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.266 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.284 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.303 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.321 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.339 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.358 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.377 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.395 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.413 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:23 smithi072 ceph-mon[33912]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:41:23.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:23 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/901621182' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:41:23.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:23 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/3321729875' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:41:23.432 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.452 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:23 smithi092 ceph-mon[33665]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:41:23.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:23 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/901621182' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:41:23.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:23 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/3321729875' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:41:23.507 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.526 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.544 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.563 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:23 smithi045 ceph-mon[30165]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:41:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:23 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/901621182' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:41:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:23 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/3321729875' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:41:23.581 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.599 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.618 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.636 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.673 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.691 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.838 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:41:23.895 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.a with skip (0.7323735613121494) > weight (0.3333333333333333) 2024-05-21T12:41:23.895 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.b with skip (0.9837620068207248) > weight (0.3333333333333333) 2024-05-21T12:41:23.895 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.c with skip (0.9537263586778564) > weight (0.3333333333333333) 2024-05-21T12:41:23.895 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.a with skip (0.43815865688176103) > weight (0.3333333333333333) 2024-05-21T12:41:23.895 INFO:tasks.nvmeof.[nvmeof.thrasher]:skipping daemon nvmeof.b with skip (0.7834702285531198) > weight (0.3333333333333333) 2024-05-21T12:41:23.896 INFO:tasks.nvmeof.[nvmeof.thrasher]:kill nvmeof.c 2024-05-21T12:41:23.896 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.c:Stopping nvmeof.nvmeof.c... 2024-05-21T12:41:23.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:41:24.227 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:41:23 smithi092 systemd[1]: Stopping Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:41:24.227 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:24 smithi092 ceph-mon[33665]: pgmap v858: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 517 op/s 2024-05-21T12:41:24.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:24 smithi072 ceph-mon[33912]: pgmap v858: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 517 op/s 2024-05-21T12:41:24.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:24 smithi045 ceph-mon[30165]: pgmap v858: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 517 op/s 2024-05-21T12:41:24.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:41:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:41:24] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log.gz 2024-05-21T12:41:24.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:41:24.876 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 138.8k 90.8k 0.0k 1.4M 908.0k 0.0k nvme2n1 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 141.6k 107.2k 0.0k 1.4M 1.0M 0.0k nvme2n10 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 133.2k 63.6k 0.0k 1.3M 636.0k 0.0k nvme2n11 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 66.8k 144.8k 0.0k 668.0k 1.4M 0.0k nvme2n12 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 114.4k 117.2k 0.0k 1.1M 1.1M 0.0k nvme2n13 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 105.6k 151.2k 0.0k 1.0M 1.5M 0.0k nvme2n14 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 103.6k 108.0k 0.0k 1.0M 1.1M 0.0k nvme2n15 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 122.4k 106.4k 0.0k 1.2M 1.0M 0.0k nvme2n16 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 99.6k 136.0k 0.0k 996.0k 1.3M 0.0k nvme2n17 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 91.2k 140.4k 0.0k 912.0k 1.4M 0.0k nvme2n18 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 110.8k 128.4k 0.0k 1.1M 1.3M 0.0k nvme2n19 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 132.8k 114.4k 0.0k 1.3M 1.1M 0.0k nvme2n2 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 115.6k 88.0k 0.0k 1.1M 880.0k 0.0k nvme2n20 2024-05-21T12:41:24.877 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 115.2k 120.8k 0.0k 1.1M 1.2M 0.0k nvme2n3 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 137.6k 92.4k 0.0k 1.3M 924.0k 0.0k nvme2n4 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 137.2k 115.6k 0.0k 1.3M 1.1M 0.0k nvme2n5 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 96.0k 126.4k 0.0k 960.0k 1.2M 0.0k nvme2n6 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 113.2k 95.2k 0.0k 1.1M 952.0k 0.0k nvme2n7 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 100.8k 144.8k 0.0k 1008.0k 1.4M 0.0k nvme2n8 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 109.6k 137.6k 0.0k 1.1M 1.3M 0.0k nvme2n9 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 120.4k 87.2k 0.0k 1.2M 872.0k 0.0k nvme5n1 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 139.2k 107.6k 0.0k 1.4M 1.1M 0.0k nvme5n10 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 92.8k 134.8k 0.0k 928.0k 1.3M 0.0k nvme5n11 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 148.8k 103.6k 0.0k 1.5M 1.0M 0.0k nvme5n12 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 110.4k 92.8k 0.0k 1.1M 928.0k 0.0k nvme5n13 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 126.0k 153.2k 0.0k 1.2M 1.5M 0.0k nvme5n14 2024-05-21T12:41:24.878 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 88.0k 86.4k 0.0k 880.0k 864.0k 0.0k nvme5n15 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 114.8k 106.8k 0.0k 1.1M 1.0M 0.0k nvme5n16 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 122.0k 98.4k 0.0k 1.2M 984.0k 0.0k nvme5n17 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 143.2k 100.8k 0.0k 1.4M 1008.0k 0.0k nvme5n18 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 102.4k 140.8k 0.0k 1.0M 1.4M 0.0k nvme5n19 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 110.0k 101.6k 0.0k 1.1M 1016.0k 0.0k nvme5n2 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 129.2k 98.8k 0.0k 1.3M 988.0k 0.0k nvme5n20 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 113.2k 125.6k 0.0k 1.1M 1.2M 0.0k nvme5n3 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 124.0k 135.2k 0.0k 1.2M 1.3M 0.0k nvme5n4 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 130.0k 89.2k 0.0k 1.3M 892.0k 0.0k nvme5n5 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 98.8k 127.2k 0.0k 988.0k 1.2M 0.0k nvme5n6 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 104.8k 107.6k 0.0k 1.0M 1.1M 0.0k nvme5n7 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 137.2k 106.4k 0.0k 1.3M 1.0M 0.0k nvme5n8 2024-05-21T12:41:24.879 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 100.0k 136.0k 0.0k 1000.0k 1.3M 0.0k nvme5n9 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 128.8k 150.4k 0.0k 1.3M 1.5M 0.0k nvme8n1 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 96.8k 124.4k 0.0k 968.0k 1.2M 0.0k nvme8n10 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 148.8k 110.0k 0.0k 1.5M 1.1M 0.0k nvme8n11 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 120.8k 105.6k 0.0k 1.2M 1.0M 0.0k nvme8n12 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 102.0k 145.2k 0.0k 1020.0k 1.4M 0.0k nvme8n13 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 138.4k 93.2k 0.0k 1.4M 932.0k 0.0k nvme8n14 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 121.6k 151.6k 0.0k 1.2M 1.5M 0.0k nvme8n15 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 98.4k 105.6k 0.0k 984.0k 1.0M 0.0k nvme8n16 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 118.4k 138.0k 0.0k 1.2M 1.3M 0.0k nvme8n17 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 115.2k 117.2k 0.0k 1.1M 1.1M 0.0k nvme8n18 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 120.4k 115.6k 0.0k 1.2M 1.1M 0.0k nvme8n19 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 120.8k 111.6k 0.0k 1.2M 1.1M 0.0k nvme8n2 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.90 151.6k 98.4k 0.0k 1.5M 984.0k 0.0k nvme8n20 2024-05-21T12:41:24.880 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 132.4k 119.2k 0.0k 1.3M 1.2M 0.0k nvme8n3 2024-05-21T12:41:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 107.2k 133.6k 0.0k 1.0M 1.3M 0.0k nvme8n4 2024-05-21T12:41:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 126.0k 161.6k 0.0k 1.2M 1.6M 0.0k nvme8n5 2024-05-21T12:41:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 122.0k 101.2k 0.0k 1.2M 1012.0k 0.0k nvme8n6 2024-05-21T12:41:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 130.0k 115.6k 0.0k 1.3M 1.1M 0.0k nvme8n7 2024-05-21T12:41:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 99.6k 107.6k 0.0k 996.0k 1.1M 0.0k nvme8n8 2024-05-21T12:41:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 8.00 140.8k 124.8k 0.0k 1.4M 1.2M 0.0k nvme8n9 2024-05-21T12:41:24.881 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:26.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:26 smithi072 ceph-mon[33912]: pgmap v859: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.7 MiB/s wr, 475 op/s 2024-05-21T12:41:26.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:26 smithi092 ceph-mon[33665]: pgmap v859: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.7 MiB/s wr, 475 op/s 2024-05-21T12:41:26.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:26 smithi045 ceph-mon[30165]: pgmap v859: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.7 MiB/s rd, 6.7 MiB/s wr, 475 op/s 2024-05-21T12:41:28.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:28 smithi072 ceph-mon[33912]: pgmap v860: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 488 op/s 2024-05-21T12:41:28.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:28 smithi092 ceph-mon[33665]: pgmap v860: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 488 op/s 2024-05-21T12:41:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:28 smithi045 ceph-mon[30165]: pgmap v860: 33 pgs: 33 active+clean; 1.0 GiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 488 op/s 2024-05-21T12:41:30.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:30 smithi072 ceph-mon[33912]: pgmap v861: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 460 op/s 2024-05-21T12:41:30.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:30 smithi092 ceph-mon[33665]: pgmap v861: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 460 op/s 2024-05-21T12:41:30.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:30 smithi045 ceph-mon[30165]: pgmap v861: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 460 op/s 2024-05-21T12:41:32.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:32 smithi072 ceph-mon[33912]: pgmap v862: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 463 op/s 2024-05-21T12:41:32.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:32 smithi092 ceph-mon[33665]: pgmap v862: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 463 op/s 2024-05-21T12:41:32.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:32 smithi045 ceph-mon[30165]: pgmap v862: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 463 op/s 2024-05-21T12:41:34.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:34 smithi072 ceph-mon[33912]: pgmap v863: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 522 op/s 2024-05-21T12:41:34.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:34 smithi092 ceph-mon[33665]: pgmap v863: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 522 op/s 2024-05-21T12:41:34.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:41:34 smithi092 bash[59739]: time="2024-05-21T12:41:34Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c in 10 seconds, resorting to SIGKILL" 2024-05-21T12:41:34.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:41:34 smithi092 podman[59739]: 2024-05-21 12:41:34.242162903 +0000 UTC m=+10.137691270 container died 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, build-date=2024-05-19 17:57:19 UTC, io.ceph.git.branch=tags/1.2.9, org.centos.version=9.0-21.el9, io.spdk.build.ceph-release=19.0.0, io.ceph.version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, distribution-scope=public, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, name=ceph-nvmeof, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.git.repo=https://github.com/ceph/spdk.git, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, url=https://github.com/ceph/ceph-nvmeof, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., com.redhat.component=ubi9-container, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.version=24.01, summary=Ceph NVMe over Fabrics Gateway, maintainer=Ceph Developers , vendor=Red Hat, Inc., io.ceph.nvmeof=None, version=1.2.9, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.component=ceph-nvmeof, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.branch=undefined, io.ceph.maintainer=Ceph Developers , io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., release=, io.spdk.build.makeflags=, io.buildah.version=1.29.0, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.pkgdep-args=--rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.openshift.wants=ceph,rbd, architecture=x86_64, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF).) 2024-05-21T12:41:34.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:34 smithi045 ceph-mon[30165]: pgmap v863: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 7.2 MiB/s wr, 522 op/s 2024-05-21T12:41:34.860 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:41:34 smithi092 podman[59739]: 2024-05-21 12:41:34.624894596 +0000 UTC m=+10.520422955 container cleanup 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.ceph-release=19.0.0, name=ceph-nvmeof, build-date=2024-05-19 17:57:19 UTC, io.buildah.version=1.29.0, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., com.redhat.component=ubi9-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, distribution-scope=public, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.version=24.01, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.openshift.wants=ceph,rbd, vendor=Red Hat, Inc., io.ceph.maintainer=Ceph Developers , release=, io.spdk.build.makeflags=, vcs-type=git, org.centos.version=9.0-21.el9, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.branch=undefined, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, maintainer=Ceph Developers , description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.component=ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, url=https://github.com/ceph/ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.nvmeof=None, version=1.2.9, architecture=x86_64, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.summary=Ceph NVMe over Fabrics Gateway) 2024-05-21T12:41:34.860 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:41:34 smithi092 bash[59739]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c 2024-05-21T12:41:34.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:34.879 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 71.2k 102.8k 0.0k 712.0k 1.0M 0.0k nvme2n1 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 112.4k 103.6k 0.0k 1.1M 1.0M 0.0k nvme2n10 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 124.0k 84.8k 0.0k 1.2M 848.0k 0.0k nvme2n11 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 132.4k 83.2k 0.0k 1.3M 832.0k 0.0k nvme2n12 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 110.0k 84.8k 0.0k 1.1M 848.0k 0.0k nvme2n13 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 107.6k 110.4k 0.0k 1.1M 1.1M 0.0k nvme2n14 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 88.4k 98.0k 0.0k 884.0k 980.0k 0.0k nvme2n15 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 84.0k 117.6k 0.0k 840.0k 1.1M 0.0k nvme2n16 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 114.0k 97.6k 0.0k 1.1M 976.0k 0.0k nvme2n17 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 103.6k 93.2k 0.0k 1.0M 932.0k 0.0k nvme2n18 2024-05-21T12:41:34.880 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 91.6k 81.6k 0.0k 916.0k 816.0k 0.0k nvme2n19 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 94.4k 107.6k 0.0k 944.0k 1.1M 0.0k nvme2n2 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 82.0k 99.2k 0.0k 820.0k 992.0k 0.0k nvme2n20 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 78.0k 132.4k 0.0k 780.0k 1.3M 0.0k nvme2n3 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 88.8k 128.4k 0.0k 888.0k 1.3M 0.0k nvme2n4 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 103.6k 84.0k 0.0k 1.0M 840.0k 0.0k nvme2n5 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 91.2k 85.2k 0.0k 912.0k 852.0k 0.0k nvme2n6 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 98.4k 109.2k 0.0k 984.0k 1.1M 0.0k nvme2n7 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 82.8k 126.0k 0.0k 828.0k 1.2M 0.0k nvme2n8 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 108.0k 74.4k 0.0k 1.1M 744.0k 0.0k nvme2n9 2024-05-21T12:41:34.881 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 70.4k 113.6k 0.0k 704.0k 1.1M 0.0k nvme5n1 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 117.2k 112.8k 0.0k 1.1M 1.1M 0.0k nvme5n10 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 97.2k 78.8k 0.0k 972.0k 788.0k 0.0k nvme5n11 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 81.6k 100.8k 0.0k 816.0k 1008.0k 0.0k nvme5n12 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 127.2k 84.8k 0.0k 1.2M 848.0k 0.0k nvme5n13 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 113.2k 100.0k 0.0k 1.1M 1000.0k 0.0k nvme5n14 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 96.0k 125.6k 0.0k 960.0k 1.2M 0.0k nvme5n15 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 116.8k 74.8k 0.0k 1.1M 748.0k 0.0k nvme5n16 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 96.8k 94.8k 0.0k 968.0k 948.0k 0.0k nvme5n17 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 91.6k 109.6k 0.0k 916.0k 1.1M 0.0k nvme5n18 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 125.2k 102.4k 0.0k 1.2M 1.0M 0.0k nvme5n19 2024-05-21T12:41:34.882 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 71.2k 125.2k 0.0k 712.0k 1.2M 0.0k nvme5n2 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 111.2k 102.0k 0.0k 1.1M 1020.0k 0.0k nvme5n20 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 100.4k 102.4k 0.0k 1004.0k 1.0M 0.0k nvme5n3 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 114.0k 100.0k 0.0k 1.1M 1000.0k 0.0k nvme5n4 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 107.6k 116.8k 0.0k 1.1M 1.1M 0.0k nvme5n5 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 118.8k 95.2k 0.0k 1.2M 952.0k 0.0k nvme5n6 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 82.0k 108.8k 0.0k 820.0k 1.1M 0.0k nvme5n7 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 107.2k 100.8k 0.0k 1.0M 1008.0k 0.0k nvme5n8 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 82.8k 109.6k 0.0k 828.0k 1.1M 0.0k nvme5n9 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 108.8k 91.2k 0.0k 1.1M 912.0k 0.0k nvme8n1 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 102.0k 74.4k 0.0k 1020.0k 744.0k 0.0k nvme8n10 2024-05-21T12:41:34.883 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 64.4k 124.4k 0.0k 644.0k 1.2M 0.0k nvme8n11 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 129.6k 58.8k 0.0k 1.3M 588.0k 0.0k nvme8n12 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.30 102.8k 101.6k 0.0k 1.0M 1016.0k 0.0k nvme8n13 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 102.4k 91.6k 0.0k 1.0M 916.0k 0.0k nvme8n14 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 108.0k 92.0k 0.0k 1.1M 920.0k 0.0k nvme8n15 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 100.0k 83.6k 0.0k 1000.0k 836.0k 0.0k nvme8n16 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 117.2k 89.2k 0.0k 1.1M 892.0k 0.0k nvme8n17 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 86.4k 117.6k 0.0k 864.0k 1.1M 0.0k nvme8n18 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 98.8k 106.0k 0.0k 988.0k 1.0M 0.0k nvme8n19 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 102.4k 108.4k 0.0k 1.0M 1.1M 0.0k nvme8n2 2024-05-21T12:41:34.884 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 98.0k 90.0k 0.0k 980.0k 900.0k 0.0k nvme8n20 2024-05-21T12:41:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 113.6k 97.2k 0.0k 1.1M 972.0k 0.0k nvme8n3 2024-05-21T12:41:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 114.0k 90.8k 0.0k 1.1M 908.0k 0.0k nvme8n4 2024-05-21T12:41:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 120.8k 91.2k 0.0k 1.2M 912.0k 0.0k nvme8n5 2024-05-21T12:41:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 107.6k 87.2k 0.0k 1.1M 872.0k 0.0k nvme8n6 2024-05-21T12:41:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 116.4k 92.8k 0.0k 1.1M 928.0k 0.0k nvme8n7 2024-05-21T12:41:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 93.2k 96.4k 0.0k 932.0k 964.0k 0.0k nvme8n8 2024-05-21T12:41:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 7.20 111.6k 80.4k 0.0k 1.1M 804.0k 0.0k nvme8n9 2024-05-21T12:41:34.885 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:35.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service' 2024-05-21T12:41:35.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:41:35.042 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.c:Stopped nvmeof.nvmeof.c 2024-05-21T12:41:35.042 INFO:tasks.nvmeof.[nvmeof.thrasher]:waiting for 122 secs before reviving 2024-05-21T12:41:36.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:36 smithi072 ceph-mon[33912]: pgmap v864: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 472 op/s 2024-05-21T12:41:36.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:36 smithi092 ceph-mon[33665]: pgmap v864: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 472 op/s 2024-05-21T12:41:36.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:36 smithi045 ceph-mon[30165]: pgmap v864: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 472 op/s 2024-05-21T12:41:38.419 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:38 smithi072 ceph-mon[33912]: pgmap v865: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-05-21T12:41:38.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:38 smithi092 ceph-mon[33665]: pgmap v865: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-05-21T12:41:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:38 smithi045 ceph-mon[30165]: pgmap v865: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 439 op/s 2024-05-21T12:41:40.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:40 smithi072 ceph-mon[33912]: pgmap v866: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 363 op/s 2024-05-21T12:41:40.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:40 smithi092 ceph-mon[33665]: pgmap v866: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 363 op/s 2024-05-21T12:41:40.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:40 smithi045 ceph-mon[30165]: pgmap v866: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 363 op/s 2024-05-21T12:41:42.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:42 smithi072 ceph-mon[33912]: pgmap v867: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2024-05-21T12:41:42.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:42 smithi092 ceph-mon[33665]: pgmap v867: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2024-05-21T12:41:42.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:42 smithi045 ceph-mon[30165]: pgmap v867: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2024-05-21T12:41:44.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:44 smithi072 ceph-mon[33912]: pgmap v868: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 248 op/s 2024-05-21T12:41:44.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:44 smithi092 ceph-mon[33665]: pgmap v868: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 248 op/s 2024-05-21T12:41:44.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:44 smithi045 ceph-mon[30165]: pgmap v868: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 248 op/s 2024-05-21T12:41:44.868 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:41:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n1 2024-05-21T12:41:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n10 2024-05-21T12:41:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n11 2024-05-21T12:41:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n12 2024-05-21T12:41:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n13 2024-05-21T12:41:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n14 2024-05-21T12:41:44.877 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n15 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n16 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n17 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n18 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n19 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n2 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n20 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n3 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n4 2024-05-21T12:41:44.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n5 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n6 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n7 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n8 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n9 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n1 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n10 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n11 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n12 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n13 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n14 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n15 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n16 2024-05-21T12:41:44.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n17 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n18 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n19 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n2 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n20 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n3 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n4 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n5 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n6 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n7 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n8 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n1 2024-05-21T12:41:44.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n10 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n11 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n12 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n13 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n14 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n15 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n16 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n17 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n18 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n19 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n2 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n20 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n3 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n4 2024-05-21T12:41:44.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n5 2024-05-21T12:41:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n6 2024-05-21T12:41:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n7 2024-05-21T12:41:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:41:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n9 2024-05-21T12:41:44.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:46.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:46 smithi072 ceph-mon[33912]: pgmap v869: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-21T12:41:46.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:46 smithi092 ceph-mon[33665]: pgmap v869: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-21T12:41:46.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:46 smithi045 ceph-mon[30165]: pgmap v869: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-21T12:41:48.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:48 smithi072 ceph-mon[33912]: pgmap v870: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 768 KiB/s rd, 646 KiB/s wr, 56 op/s 2024-05-21T12:41:48.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:48 smithi072 ceph-mon[33912]: osdmap e143: 8 total, 8 up, 8 in 2024-05-21T12:41:48.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:48 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:48 smithi092 ceph-mon[33665]: pgmap v870: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 768 KiB/s rd, 646 KiB/s wr, 56 op/s 2024-05-21T12:41:48.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:48 smithi092 ceph-mon[33665]: osdmap e143: 8 total, 8 up, 8 in 2024-05-21T12:41:48.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:48 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:48.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-mon[30165]: pgmap v870: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 768 KiB/s rd, 646 KiB/s wr, 56 op/s 2024-05-21T12:41:48.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-mon[30165]: osdmap e143: 8 total, 8 up, 8 in 2024-05-21T12:41:48.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:41:48] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:41:48.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:41:48.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:41:48.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:41:48.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:41:48.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:41:48.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:41:48.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: ], , client address: IPv4 172.21.15.45:46006 2024-05-21T12:41:48.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:48 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:41:48.080052] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000251 seconds 2024-05-21T12:41:49.566 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:41:49] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:41:49.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:41:49.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:41:49.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:41:49.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: ], , client address: IPv4 172.21.15.45:46018 2024-05-21T12:41:49.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:41:49.136102] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000236 seconds 2024-05-21T12:41:50.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:50 smithi072 ceph-mon[33912]: pgmap v872: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:41:50.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:50 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:50.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:50 smithi072 ceph-mon[33912]: osdmap e144: 8 total, 8 up, 8 in 2024-05-21T12:41:50.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:50 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:50 smithi092 ceph-mon[33665]: pgmap v872: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:41:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:50 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:50 smithi092 ceph-mon[33665]: osdmap e144: 8 total, 8 up, 8 in 2024-05-21T12:41:50.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:50 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:50 smithi045 ceph-mon[30165]: pgmap v872: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:41:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:50 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:50 smithi045 ceph-mon[30165]: osdmap e144: 8 total, 8 up, 8 in 2024-05-21T12:41:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:50 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:51.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:51 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:51.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:51 smithi072 ceph-mon[33912]: osdmap e145: 8 total, 8 up, 8 in 2024-05-21T12:41:51.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:51 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:51.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:51 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:51.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:51 smithi092 ceph-mon[33665]: osdmap e145: 8 total, 8 up, 8 in 2024-05-21T12:41:51.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:51 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:51.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:51 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:51.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:51 smithi045 ceph-mon[30165]: osdmap e145: 8 total, 8 up, 8 in 2024-05-21T12:41:51.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:51 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:52.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:52 smithi072 ceph-mon[33912]: pgmap v875: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 16 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-21T12:41:52.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:52 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:52.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:52 smithi072 ceph-mon[33912]: osdmap e146: 8 total, 8 up, 8 in 2024-05-21T12:41:52.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:52 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:52.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:52 smithi092 ceph-mon[33665]: pgmap v875: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 16 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-21T12:41:52.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:52 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:52.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:52 smithi092 ceph-mon[33665]: osdmap e146: 8 total, 8 up, 8 in 2024-05-21T12:41:52.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:52 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:52.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:52 smithi045 ceph-mon[30165]: pgmap v875: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 16 KiB/s rd, 170 B/s wr, 3 op/s 2024-05-21T12:41:52.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:52 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:52.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:52 smithi045 ceph-mon[30165]: osdmap e146: 8 total, 8 up, 8 in 2024-05-21T12:41:52.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:52 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:53.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:53 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:53 smithi072 ceph-mon[33912]: osdmap e147: 8 total, 8 up, 8 in 2024-05-21T12:41:53.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:53 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:53.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:53 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:53 smithi092 ceph-mon[33665]: osdmap e147: 8 total, 8 up, 8 in 2024-05-21T12:41:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:53 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:53.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:53 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:53.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:53 smithi045 ceph-mon[30165]: osdmap e147: 8 total, 8 up, 8 in 2024-05-21T12:41:53.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:53 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:54.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:54 smithi092 ceph-mon[33665]: pgmap v878: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 59 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-05-21T12:41:54.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:54 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:54.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:54 smithi092 ceph-mon[33665]: osdmap e148: 8 total, 8 up, 8 in 2024-05-21T12:41:54.404 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:54 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:54.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:54 smithi072 ceph-mon[33912]: pgmap v878: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 59 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-05-21T12:41:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:54 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:54 smithi072 ceph-mon[33912]: osdmap e148: 8 total, 8 up, 8 in 2024-05-21T12:41:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:54 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:54.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:54 smithi045 ceph-mon[30165]: pgmap v878: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 59 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-05-21T12:41:54.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:54 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:54.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:54 smithi045 ceph-mon[30165]: osdmap e148: 8 total, 8 up, 8 in 2024-05-21T12:41:54.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:54 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:54.869 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:54.877 INFO:tasks.workunit.client.3.smithi137.stdout: tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme2n1 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n10 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n11 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n12 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n13 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme2n14 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme2n15 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme2n16 2024-05-21T12:41:54.878 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme2n17 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.0k 0.0k 0.0k 20.0k 0.0k nvme2n18 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme2n19 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme2n2 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.6k 0.0k 0.0k 36.0k 0.0k nvme2n20 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 6.0k 0.0k 0.0k 60.0k 0.0k 0.0k nvme2n3 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme2n4 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 3.2k 0.0k 0.0k 32.0k 0.0k nvme2n5 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n6 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n7 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n8 2024-05-21T12:41:54.879 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme2n9 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 5.6k 0.0k 0.0k 56.0k 0.0k nvme5n1 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n10 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n11 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n12 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n13 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.2k 0.0k 0.0k 32.0k 0.0k 0.0k nvme5n14 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.8k 0.0k 0.0k 48.0k 0.0k nvme5n15 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 6.0k 0.0k 0.0k 60.0k 0.0k nvme5n16 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme5n17 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.0k 0.0k 0.0k 40.0k 0.0k 0.0k nvme5n18 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.4k 0.0k 0.0k 44.0k 0.0k 0.0k nvme5n19 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme5n2 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.4k 0.0k 0.0k 24.0k 0.0k 0.0k nvme5n20 2024-05-21T12:41:54.880 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.8k 0.0k 0.0k 8.0k 0.0k 0.0k nvme5n3 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 2.4k 0.0k 0.0k 24.0k 0.0k nvme5n4 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme5n5 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n6 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n7 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n8 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme5n9 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n1 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n10 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n11 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n12 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.6k 0.0k 0.0k 16.0k 0.0k nvme8n13 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 4.0k 0.0k 0.0k 40.0k 0.0k nvme8n14 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n15 2024-05-21T12:41:54.881 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 1.6k 0.0k 0.0k 16.0k 0.0k 0.0k nvme8n16 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme8n17 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 1.2k 0.0k 0.0k 12.0k 0.0k nvme8n18 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.0k 0.8k 0.0k 0.0k 8.0k 0.0k nvme8n19 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 2.0k 0.0k 0.0k 20.0k 0.0k 0.0k nvme8n2 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 0.4k 0.0k 0.0k 4.0k 0.0k 0.0k nvme8n20 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 3.6k 0.0k 0.0k 36.0k 0.0k 0.0k nvme8n3 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.10 4.4k 0.0k 0.0k 44.0k 0.0k 0.0k nvme8n4 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n5 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n6 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n7 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n8 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 0.00 0.0k 0.0k 0.0k 0.0k 0.0k 0.0k nvme8n9 2024-05-21T12:41:54.882 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:54.883 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:41:55.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:55 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:55.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:55 smithi072 ceph-mon[33912]: osdmap e149: 8 total, 8 up, 8 in 2024-05-21T12:41:55.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:55 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:55.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:55 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:55.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:55 smithi092 ceph-mon[33665]: osdmap e149: 8 total, 8 up, 8 in 2024-05-21T12:41:55.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:55 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:55 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:55 smithi045 ceph-mon[30165]: osdmap e149: 8 total, 8 up, 8 in 2024-05-21T12:41:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:55 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:56.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:56 smithi072 ceph-mon[33912]: pgmap v881: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 105 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-05-21T12:41:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:56 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:56 smithi072 ceph-mon[33912]: osdmap e150: 8 total, 8 up, 8 in 2024-05-21T12:41:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:56 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:56 smithi092 ceph-mon[33665]: pgmap v881: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 105 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-05-21T12:41:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:56 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:56 smithi092 ceph-mon[33665]: osdmap e150: 8 total, 8 up, 8 in 2024-05-21T12:41:56.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:56 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:56 smithi045 ceph-mon[30165]: pgmap v881: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 105 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-05-21T12:41:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:56 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:56 smithi045 ceph-mon[30165]: osdmap e150: 8 total, 8 up, 8 in 2024-05-21T12:41:56.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:56 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:57 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:57 smithi072 ceph-mon[33912]: osdmap e151: 8 total, 8 up, 8 in 2024-05-21T12:41:57.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:57 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:57.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:57 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:57 smithi092 ceph-mon[33665]: osdmap e151: 8 total, 8 up, 8 in 2024-05-21T12:41:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:57 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:57 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:57 smithi045 ceph-mon[30165]: osdmap e151: 8 total, 8 up, 8 in 2024-05-21T12:41:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:57 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:58.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:58 smithi072 ceph-mon[33912]: pgmap v884: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 163 KiB/s rd, 74 KiB/s wr, 28 op/s 2024-05-21T12:41:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:58 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:58 smithi072 ceph-mon[33912]: osdmap e152: 8 total, 8 up, 8 in 2024-05-21T12:41:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:58 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:58.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:58 smithi092 ceph-mon[33665]: pgmap v884: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 163 KiB/s rd, 74 KiB/s wr, 28 op/s 2024-05-21T12:41:58.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:58 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:58.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:58 smithi092 ceph-mon[33665]: osdmap e152: 8 total, 8 up, 8 in 2024-05-21T12:41:58.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:58 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:58 smithi045 ceph-mon[30165]: pgmap v884: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 163 KiB/s rd, 74 KiB/s wr, 28 op/s 2024-05-21T12:41:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:58 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:58 smithi045 ceph-mon[30165]: osdmap e152: 8 total, 8 up, 8 in 2024-05-21T12:41:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:58 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:59.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:59 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:59.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:59 smithi072 ceph-mon[33912]: osdmap e153: 8 total, 8 up, 8 in 2024-05-21T12:41:59.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:41:59 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:59 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:59 smithi092 ceph-mon[33665]: osdmap e153: 8 total, 8 up, 8 in 2024-05-21T12:41:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:41:59 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:41:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:59 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:41:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:59 smithi045 ceph-mon[30165]: osdmap e153: 8 total, 8 up, 8 in 2024-05-21T12:41:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:41:59 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:00.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:00 smithi072 ceph-mon[33912]: pgmap v887: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 97 KiB/s wr, 28 op/s 2024-05-21T12:42:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:00 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:00 smithi072 ceph-mon[33912]: osdmap e154: 8 total, 8 up, 8 in 2024-05-21T12:42:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:00 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:00 smithi092 ceph-mon[33665]: pgmap v887: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 97 KiB/s wr, 28 op/s 2024-05-21T12:42:00.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:00 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:00.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:00 smithi092 ceph-mon[33665]: osdmap e154: 8 total, 8 up, 8 in 2024-05-21T12:42:00.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:00 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:00 smithi045 ceph-mon[30165]: pgmap v887: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 144 KiB/s rd, 97 KiB/s wr, 28 op/s 2024-05-21T12:42:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:00 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:00 smithi045 ceph-mon[30165]: osdmap e154: 8 total, 8 up, 8 in 2024-05-21T12:42:00.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:00 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:01.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:01 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:01.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:01 smithi072 ceph-mon[33912]: osdmap e155: 8 total, 8 up, 8 in 2024-05-21T12:42:01.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:01 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:01.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:01 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:01.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:01 smithi092 ceph-mon[33665]: osdmap e155: 8 total, 8 up, 8 in 2024-05-21T12:42:01.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:01 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:01 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:01 smithi045 ceph-mon[30165]: osdmap e155: 8 total, 8 up, 8 in 2024-05-21T12:42:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:01 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:02 smithi072 ceph-mon[33912]: pgmap v890: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-05-21T12:42:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:02 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:02 smithi072 ceph-mon[33912]: osdmap e156: 8 total, 8 up, 8 in 2024-05-21T12:42:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:02 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:02 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:42:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:02 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:42:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:02 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:42:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:02 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:42:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:02 smithi092 ceph-mon[33665]: pgmap v890: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-05-21T12:42:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:02 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:02 smithi092 ceph-mon[33665]: osdmap e156: 8 total, 8 up, 8 in 2024-05-21T12:42:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:02 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:02 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:42:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:02 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:42:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:02 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:42:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:02 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:42:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:02 smithi045 ceph-mon[30165]: pgmap v890: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 79 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-05-21T12:42:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:02 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:02 smithi045 ceph-mon[30165]: osdmap e156: 8 total, 8 up, 8 in 2024-05-21T12:42:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:02 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:02.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:42:02.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:42:02.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:42:02.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:02 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:42:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:03 smithi072 ceph-mon[33912]: pgmap v892: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 143 KiB/s rd, 144 KiB/s wr, 32 op/s 2024-05-21T12:42:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:03 smithi072 ceph-mon[33912]: pgmap v893: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 169 KiB/s rd, 175 KiB/s wr, 37 op/s 2024-05-21T12:42:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:03 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:03 smithi072 ceph-mon[33912]: osdmap e157: 8 total, 8 up, 8 in 2024-05-21T12:42:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:03 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:03.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:03 smithi092 ceph-mon[33665]: pgmap v892: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 143 KiB/s rd, 144 KiB/s wr, 32 op/s 2024-05-21T12:42:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:03 smithi092 ceph-mon[33665]: pgmap v893: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 169 KiB/s rd, 175 KiB/s wr, 37 op/s 2024-05-21T12:42:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:03 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:03 smithi092 ceph-mon[33665]: osdmap e157: 8 total, 8 up, 8 in 2024-05-21T12:42:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:03 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:03 smithi045 ceph-mon[30165]: pgmap v892: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 143 KiB/s rd, 144 KiB/s wr, 32 op/s 2024-05-21T12:42:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:03 smithi045 ceph-mon[30165]: pgmap v893: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 169 KiB/s rd, 175 KiB/s wr, 37 op/s 2024-05-21T12:42:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:03 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:03.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:03 smithi045 ceph-mon[30165]: osdmap e157: 8 total, 8 up, 8 in 2024-05-21T12:42:03.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:03 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:04.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:04 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:04.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:04 smithi072 ceph-mon[33912]: osdmap e158: 8 total, 8 up, 8 in 2024-05-21T12:42:04.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:04 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:04.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:04 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:04 smithi092 ceph-mon[33665]: osdmap e158: 8 total, 8 up, 8 in 2024-05-21T12:42:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:04 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:04 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:04 smithi045 ceph-mon[30165]: osdmap e158: 8 total, 8 up, 8 in 2024-05-21T12:42:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:04 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:05 smithi092 ceph-mon[33665]: pgmap v896: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 208 KiB/s wr, 36 op/s 2024-05-21T12:42:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:05 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:05 smithi092 ceph-mon[33665]: osdmap e159: 8 total, 8 up, 8 in 2024-05-21T12:42:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:05 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:05 smithi045 ceph-mon[30165]: pgmap v896: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 208 KiB/s wr, 36 op/s 2024-05-21T12:42:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:05 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:05 smithi045 ceph-mon[30165]: osdmap e159: 8 total, 8 up, 8 in 2024-05-21T12:42:05.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:05 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:05.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:05 smithi072 ceph-mon[33912]: pgmap v896: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 186 KiB/s rd, 208 KiB/s wr, 36 op/s 2024-05-21T12:42:05.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:05 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:05.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:05 smithi072 ceph-mon[33912]: osdmap e159: 8 total, 8 up, 8 in 2024-05-21T12:42:05.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:05 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:06.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:06 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:06.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:06 smithi092 ceph-mon[33665]: osdmap e160: 8 total, 8 up, 8 in 2024-05-21T12:42:06.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:06 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:06 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:06 smithi045 ceph-mon[30165]: osdmap e160: 8 total, 8 up, 8 in 2024-05-21T12:42:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:06 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:06.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:06 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:06.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:06 smithi072 ceph-mon[33912]: osdmap e160: 8 total, 8 up, 8 in 2024-05-21T12:42:06.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:06 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:07 smithi092 ceph-mon[33665]: pgmap v899: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 197 KiB/s rd, 179 KiB/s wr, 32 op/s 2024-05-21T12:42:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:07 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:07 smithi092 ceph-mon[33665]: osdmap e161: 8 total, 8 up, 8 in 2024-05-21T12:42:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:07 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:07 smithi045 ceph-mon[30165]: pgmap v899: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 197 KiB/s rd, 179 KiB/s wr, 32 op/s 2024-05-21T12:42:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:07 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:07 smithi045 ceph-mon[30165]: osdmap e161: 8 total, 8 up, 8 in 2024-05-21T12:42:07.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:07 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:07.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:07 smithi072 ceph-mon[33912]: pgmap v899: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 197 KiB/s rd, 179 KiB/s wr, 32 op/s 2024-05-21T12:42:07.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:07 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:07.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:07 smithi072 ceph-mon[33912]: osdmap e161: 8 total, 8 up, 8 in 2024-05-21T12:42:07.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:07 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:08.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:08 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:08 smithi092 ceph-mon[33665]: osdmap e162: 8 total, 8 up, 8 in 2024-05-21T12:42:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:08 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:08 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:08 smithi045 ceph-mon[30165]: osdmap e162: 8 total, 8 up, 8 in 2024-05-21T12:42:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:08 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:08.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:08 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:08.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:08 smithi072 ceph-mon[33912]: osdmap e162: 8 total, 8 up, 8 in 2024-05-21T12:42:08.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:08 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd={"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"} : dispatch 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout:nvmeof-fio-test: (groupid=0, jobs=1): err= 0: pid=49664: Tue May 21 12:42:09 2024 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout: read: IOPS=227, BW=5604KiB/s (5738kB/s)(3283MiB/600003msec) 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout: clat (usec): min=82, max=7998.6k, avg=777.42, stdev=21633.65 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout: lat (usec): min=82, max=7998.6k, avg=777.75, stdev=21633.65 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout: clat percentiles (usec): 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout: | 1.00th=[ 416], 5.00th=[ 474], 10.00th=[ 510], 20.00th=[ 570], 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout: | 30.00th=[ 611], 40.00th=[ 652], 50.00th=[ 693], 60.00th=[ 725], 2024-05-21T12:42:09.020 INFO:tasks.workunit.client.3.smithi137.stdout: | 70.00th=[ 775], 80.00th=[ 824], 90.00th=[ 906], 95.00th=[ 988], 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: | 99.00th=[ 2008], 99.50th=[ 2573], 99.90th=[ 2900], 99.95th=[ 2966], 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: | 99.99th=[ 5604] 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: bw ( KiB/s): min= 8, max=17544, per=100.00%, avg=7237.98, stdev=2951.35, samples=929 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: iops : min= 2, max= 550, avg=294.23, stdev=109.84, samples=929 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: write: IOPS=228, BW=5599KiB/s (5733kB/s)(3281MiB/600003msec); 0 zone resets 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: clat (usec): min=1026, max=27530k, avg=3519.26, stdev=95578.56 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: lat (usec): min=1036, max=27530k, avg=3598.26, stdev=95578.70 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: clat percentiles (usec): 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: | 1.00th=[ 1254], 5.00th=[ 1401], 10.00th=[ 1516], 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: | 20.00th=[ 1696], 30.00th=[ 1860], 40.00th=[ 2024], 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: | 50.00th=[ 2212], 60.00th=[ 2409], 70.00th=[ 2606], 2024-05-21T12:42:09.021 INFO:tasks.workunit.client.3.smithi137.stdout: | 80.00th=[ 2868], 90.00th=[ 3261], 95.00th=[ 3589], 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: | 99.00th=[ 4359], 99.50th=[ 4686], 99.90th=[ 17957], 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: | 99.95th=[ 994051], 99.99th=[1010828] 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: bw ( KiB/s): min= 8, max=16048, per=100.00%, avg=6991.69, stdev=3106.81, samples=961 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: iops : min= 1, max= 530, avg=284.67, stdev=116.54, samples=961 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: lat (usec) : 100=0.01%, 250=0.08%, 500=4.26%, 750=28.23%, 1000=15.19% 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: lat (msec) : 2=20.85%, 4=30.33%, 10=0.97%, 20=0.03%, 50=0.01% 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: lat (msec) : 100=0.01%, 500=0.01%, 1000=0.02%, 2000=0.02%, >=2000=0.01% 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: cpu : usr=2.11%, sys=0.92%, ctx=273744, majf=0, minf=3819 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% 2024-05-21T12:42:09.022 INFO:tasks.workunit.client.3.smithi137.stdout: issued rwts: total=136695,136803,0,0 short=0,0,0,0 dropped=0,0,0,0 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: latency : target=0, window=0, percentile=100.00%, depth=1 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout:Run status group 0 (all jobs): 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: READ: bw=5604KiB/s (5738kB/s), 5604KiB/s-5604KiB/s (5738kB/s-5738kB/s), io=3283MiB (3443MB), run=600003-600003msec 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: WRITE: bw=5599KiB/s (5733kB/s), 5599KiB/s-5599KiB/s (5733kB/s-5733kB/s), io=3281MiB (3440MB), run=600003-600003msec 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout:Disk stats (read/write): 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n1: ios=2308/2332, merge=0/0, ticks=1649/7438, in_queue=9087, util=2.26% 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n1: ios=2359/2285, merge=0/0, ticks=1740/5916, in_queue=7656, util=2.03% 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n1: ios=2357/2291, merge=0/0, ticks=1651/7076, in_queue=8727, util=2.20% 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n2: ios=2406/2237, merge=0/0, ticks=1653/6444, in_queue=8097, util=2.10% 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n2: ios=2367/2285, merge=0/0, ticks=1712/7490, in_queue=9202, util=2.28% 2024-05-21T12:42:09.023 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n2: ios=2365/2280, merge=0/0, ticks=1613/7201, in_queue=8814, util=2.22% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n3: ios=2389/2260, merge=0/0, ticks=1696/6808, in_queue=8504, util=2.17% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n3: ios=2381/2264, merge=0/0, ticks=1638/6767, in_queue=8405, util=2.15% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n3: ios=2386/2261, merge=0/0, ticks=1636/5739, in_queue=7375, util=1.98% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n4: ios=2335/2314, merge=0/0, ticks=1731/7803, in_queue=9534, util=2.34% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n4: ios=2332/2308, merge=0/0, ticks=1650/6200, in_queue=7850, util=2.05% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n4: ios=2370/2273, merge=0/0, ticks=1666/7437, in_queue=9103, util=2.27% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n5: ios=2415/2236, merge=0/0, ticks=1707/6404, in_queue=8111, util=2.10% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n5: ios=2395/2260, merge=0/0, ticks=1695/7281, in_queue=8976, util=2.24% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n5: ios=2382/2265, merge=0/0, ticks=1666/7101, in_queue=8767, util=2.21% 2024-05-21T12:42:09.024 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n6: ios=2346/2303, merge=0/0, ticks=1667/7259, in_queue=8926, util=2.24% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n6: ios=2378/2262, merge=0/0, ticks=1743/7627, in_queue=9370, util=2.31% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n6: ios=2388/2254, merge=0/0, ticks=1654/5974, in_queue=7628, util=2.03% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n7: ios=2419/2229, merge=0/0, ticks=1770/7351, in_queue=9121, util=2.27% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n7: ios=2362/2285, merge=0/0, ticks=1677/6321, in_queue=7998, util=2.08% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n7: ios=2367/2277, merge=0/0, ticks=1666/7239, in_queue=8905, util=2.24% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n8: ios=2368/2281, merge=0/0, ticks=1653/5964, in_queue=7617, util=2.02% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n8: ios=2340/2310, merge=0/0, ticks=1642/7136, in_queue=8778, util=2.21% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n8: ios=2389/2255, merge=0/0, ticks=1739/6724, in_queue=8463, util=2.16% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n9: ios=2318/2331, merge=0/0, ticks=1595/7042, in_queue=8637, util=2.20% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n9: ios=2423/2220, merge=0/0, ticks=1736/7031, in_queue=8767, util=40.61% 2024-05-21T12:42:09.025 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n9: ios=2401/2241, merge=0/0, ticks=1680/5745, in_queue=7425, util=1.99% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n10: ios=2329/2323, merge=0/0, ticks=1690/7485, in_queue=9175, util=2.28% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n10: ios=2320/2330, merge=0/0, ticks=1622/6302, in_queue=7924, util=2.08% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n10: ios=2397/2256, merge=0/0, ticks=1731/6892, in_queue=8623, util=2.19% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n11: ios=2318/2330, merge=0/0, ticks=1608/6187, in_queue=7795, util=2.06% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n11: ios=2374/2277, merge=0/0, ticks=1639/6911, in_queue=8550, util=2.18% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n11: ios=2381/2264, merge=0/0, ticks=1660/7313, in_queue=8973, util=2.25% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n12: ios=2367/2282, merge=0/0, ticks=1610/7064, in_queue=8674, util=2.20% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n12: ios=2353/2291, merge=0/0, ticks=1656/7239, in_queue=8895, util=2.24% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n12: ios=2336/2315, merge=0/0, ticks=1627/5845, in_queue=7472, util=2.00% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n13: ios=2375/2274, merge=0/0, ticks=1637/7664, in_queue=9301, util=2.31% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n13: ios=2402/2252, merge=0/0, ticks=1578/6215, in_queue=7793, util=2.06% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n13: ios=2391/2260, merge=0/0, ticks=1730/7447, in_queue=9177, util=7.69% 2024-05-21T12:42:09.026 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n14: ios=2348/2295, merge=0/0, ticks=1619/6269, in_queue=7888, util=2.07% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n14: ios=2362/2280, merge=0/0, ticks=1621/7035, in_queue=8656, util=2.20% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n14: ios=2417/2231, merge=0/0, ticks=1665/6871, in_queue=8536, util=2.18% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n15: ios=2430/2226, merge=0/0, ticks=1640/7153, in_queue=8793, util=2.23% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n15: ios=2355/2291, merge=0/0, ticks=1712/7503, in_queue=9215, util=2.29% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n15: ios=2320/2318, merge=0/0, ticks=1630/6562, in_queue=8192, util=2.12% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n16: ios=2337/2309, merge=0/0, ticks=1583/7418, in_queue=9001, util=2.26% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n16: ios=2412/2240, merge=0/0, ticks=1704/5992, in_queue=7696, util=2.04% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n16: ios=2460/2192, merge=0/0, ticks=1722/6634, in_queue=8356, util=2.15% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n17: ios=2331/2308, merge=0/0, ticks=1586/5903, in_queue=7489, util=2.01% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n17: ios=2359/2283, merge=0/0, ticks=1685/7680, in_queue=9365, util=2.32% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n17: ios=2333/2320, merge=0/0, ticks=1683/7684, in_queue=9367, util=24.72% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n18: ios=2353/2296, merge=0/0, ticks=1545/6709, in_queue=8254, util=2.14% 2024-05-21T12:42:09.027 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n18: ios=2357/2298, merge=0/0, ticks=1723/7489, in_queue=9212, util=2.29% 2024-05-21T12:42:09.028 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n18: ios=2328/2322, merge=0/0, ticks=1610/6696, in_queue=8306, util=2.14% 2024-05-21T12:42:09.028 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n19: ios=2362/2292, merge=0/0, ticks=1570/7621, in_queue=9191, util=2.29% 2024-05-21T12:42:09.028 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n19: ios=2374/2272, merge=0/0, ticks=1626/5831, in_queue=7457, util=2.00% 2024-05-21T12:42:09.028 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n19: ios=2364/2283, merge=0/0, ticks=1660/7593, in_queue=9253, util=2.30% 2024-05-21T12:42:09.028 INFO:tasks.workunit.client.3.smithi137.stdout: nvme5n20: ios=2286/2352, merge=0/0, ticks=1656/6671, in_queue=8327, util=2.15% 2024-05-21T12:42:09.028 INFO:tasks.workunit.client.3.smithi137.stdout: nvme2n20: ios=2377/2272, merge=0/0, ticks=1562/7185, in_queue=8747, util=2.22% 2024-05-21T12:42:09.028 INFO:tasks.workunit.client.3.smithi137.stdout: nvme8n20: ios=2385/2262, merge=0/0, ticks=1775/7332, in_queue=9107, util=2.28% 2024-05-21T12:42:09.028 INFO:tasks.workunit.client.3.smithi137.stderr:+ wait 2024-05-21T12:42:09.029 INFO:tasks.workunit.client.3.smithi137.stdout:[nvmeof.fio] fio test successful! 2024-05-21T12:42:09.030 INFO:tasks.workunit.client.3.smithi137.stderr:+ echo '[nvmeof.fio] fio test successful!' 2024-05-21T12:42:09.030 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-21T12:42:09.030 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.3/client.3/tmp 2024-05-21T12:42:09.063 INFO:tasks.workunit:Stopping ['rbd/nvmeof_fio_test.sh --rbd_iostat'] on client.3... 2024-05-21T12:42:09.064 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.3 /home/ubuntu/cephtest/clone.client.3 2024-05-21T12:42:09.414 DEBUG:teuthology.parallel:result is None 2024-05-21T12:42:09.415 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.3/client.3 2024-05-21T12:42:09.441 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.3/client.3 2024-05-21T12:42:09.441 DEBUG:teuthology.orchestra.run.smithi137:> rmdir -- /home/ubuntu/cephtest/mnt.3 2024-05-21T12:42:09.496 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.3/client.3 2024-05-21T12:42:09.496 DEBUG:teuthology.run_tasks:Unwinding manager nvmeof.thrash 2024-05-21T12:42:09.508 INFO:tasks.nvmeof:joining nvmeof.thrash 2024-05-21T12:42:09.508 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:09 smithi092 ceph-mon[33665]: pgmap v902: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 501 KiB/s rd, 449 KiB/s wr, 62 op/s 2024-05-21T12:42:09.508 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:09 smithi092 ceph-mon[33665]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:09.508 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:09 smithi092 ceph-mon[33665]: osdmap e163: 8 total, 8 up, 8 in 2024-05-21T12:42:09.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:09 smithi045 ceph-mon[30165]: pgmap v902: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 501 KiB/s rd, 449 KiB/s wr, 62 op/s 2024-05-21T12:42:09.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:09 smithi045 ceph-mon[30165]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:09.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:09 smithi045 ceph-mon[30165]: osdmap e163: 8 total, 8 up, 8 in 2024-05-21T12:42:09.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:09 smithi072 ceph-mon[33912]: pgmap v902: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 501 KiB/s rd, 449 KiB/s wr, 62 op/s 2024-05-21T12:42:09.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:09 smithi072 ceph-mon[33912]: from='client.? 172.21.15.45:0/1400773162' entity='client.nvmeof.nvmeof.a' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "172.21.15.92:0/790513900"}]': finished 2024-05-21T12:42:09.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:09 smithi072 ceph-mon[33912]: osdmap e163: 8 total, 8 up, 8 in 2024-05-21T12:42:11.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:11 smithi092 ceph-mon[33665]: pgmap v904: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 679 KiB/s wr, 136 op/s 2024-05-21T12:42:11.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:11 smithi045 ceph-mon[30165]: pgmap v904: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 679 KiB/s wr, 136 op/s 2024-05-21T12:42:11.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:11 smithi072 ceph-mon[33912]: pgmap v904: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 679 KiB/s wr, 136 op/s 2024-05-21T12:42:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:13 smithi092 ceph-mon[33665]: pgmap v905: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 911 KiB/s wr, 227 op/s 2024-05-21T12:42:13.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:13 smithi045 ceph-mon[30165]: pgmap v905: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 911 KiB/s wr, 227 op/s 2024-05-21T12:42:13.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:13 smithi072 ceph-mon[33912]: pgmap v905: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 911 KiB/s wr, 227 op/s 2024-05-21T12:42:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:15 smithi092 ceph-mon[33665]: pgmap v906: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.8 MiB/s rd, 828 KiB/s wr, 245 op/s 2024-05-21T12:42:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:15 smithi045 ceph-mon[30165]: pgmap v906: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.8 MiB/s rd, 828 KiB/s wr, 245 op/s 2024-05-21T12:42:15.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:15 smithi072 ceph-mon[33912]: pgmap v906: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.8 MiB/s rd, 828 KiB/s wr, 245 op/s 2024-05-21T12:42:17.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:17 smithi092 ceph-mon[33665]: pgmap v907: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 632 KiB/s wr, 207 op/s 2024-05-21T12:42:17.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:17 smithi045 ceph-mon[30165]: pgmap v907: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 632 KiB/s wr, 207 op/s 2024-05-21T12:42:17.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:17 smithi072 ceph-mon[33912]: pgmap v907: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 4.1 MiB/s rd, 632 KiB/s wr, 207 op/s 2024-05-21T12:42:19.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:19 smithi092 ceph-mon[33665]: pgmap v908: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 522 KiB/s wr, 178 op/s 2024-05-21T12:42:19.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:19 smithi045 ceph-mon[30165]: pgmap v908: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 522 KiB/s wr, 178 op/s 2024-05-21T12:42:19.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:19 smithi072 ceph-mon[33912]: pgmap v908: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.6 MiB/s rd, 522 KiB/s wr, 178 op/s 2024-05-21T12:42:21.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:21 smithi092 ceph-mon[33665]: pgmap v909: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 439 KiB/s wr, 150 op/s 2024-05-21T12:42:21.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:21 smithi045 ceph-mon[30165]: pgmap v909: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 439 KiB/s wr, 150 op/s 2024-05-21T12:42:21.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:21 smithi072 ceph-mon[33912]: pgmap v909: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.0 MiB/s rd, 439 KiB/s wr, 150 op/s 2024-05-21T12:42:23.468 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:23 smithi072 ceph-mon[33912]: pgmap v910: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 267 KiB/s wr, 107 op/s 2024-05-21T12:42:23.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:23 smithi092 ceph-mon[33665]: pgmap v910: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 267 KiB/s wr, 107 op/s 2024-05-21T12:42:23.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:23 smithi045 ceph-mon[30165]: pgmap v910: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.3 MiB/s rd, 267 KiB/s wr, 107 op/s 2024-05-21T12:42:25.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:25 smithi092 ceph-mon[33665]: pgmap v911: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 89 KiB/s wr, 49 op/s 2024-05-21T12:42:25.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:25 smithi045 ceph-mon[30165]: pgmap v911: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 89 KiB/s wr, 49 op/s 2024-05-21T12:42:25.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:25 smithi072 ceph-mon[33912]: pgmap v911: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 MiB/s rd, 89 KiB/s wr, 49 op/s 2024-05-21T12:42:27.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:27 smithi092 ceph-mon[33665]: pgmap v912: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:27.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:27 smithi045 ceph-mon[30165]: pgmap v912: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:27.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:27 smithi072 ceph-mon[33912]: pgmap v912: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:29.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:29 smithi092 ceph-mon[33665]: pgmap v913: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:29.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:29 smithi045 ceph-mon[30165]: pgmap v913: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:29.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:29 smithi072 ceph-mon[33912]: pgmap v913: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:31.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:31 smithi092 ceph-mon[33665]: pgmap v914: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:31.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:31 smithi045 ceph-mon[30165]: pgmap v914: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:31.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:31 smithi072 ceph-mon[33912]: pgmap v914: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:33.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:33 smithi092 ceph-mon[33665]: pgmap v915: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:33.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:33 smithi045 ceph-mon[30165]: pgmap v915: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:33.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:33 smithi072 ceph-mon[33912]: pgmap v915: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:35.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:35 smithi092 ceph-mon[33665]: pgmap v916: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:35.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:35 smithi045 ceph-mon[30165]: pgmap v916: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:35.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:35 smithi072 ceph-mon[33912]: pgmap v916: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:37.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:37 smithi092 ceph-mon[33665]: pgmap v917: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:37.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:37 smithi045 ceph-mon[30165]: pgmap v917: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:37.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:37 smithi072 ceph-mon[33912]: pgmap v917: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:39.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:39 smithi092 ceph-mon[33665]: pgmap v918: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:39.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:39 smithi045 ceph-mon[30165]: pgmap v918: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:39.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:39 smithi072 ceph-mon[33912]: pgmap v918: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:41.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:41 smithi092 ceph-mon[33665]: pgmap v919: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:41.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:41 smithi045 ceph-mon[30165]: pgmap v919: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:41.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:41 smithi072 ceph-mon[33912]: pgmap v919: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:43.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:43 smithi092 ceph-mon[33665]: pgmap v920: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:43.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:43 smithi045 ceph-mon[30165]: pgmap v920: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:43.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:43 smithi072 ceph-mon[33912]: pgmap v920: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:45.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:45 smithi092 ceph-mon[33665]: pgmap v921: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:45.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:45 smithi045 ceph-mon[30165]: pgmap v921: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:45.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:45 smithi072 ceph-mon[33912]: pgmap v921: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:47.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:47 smithi092 ceph-mon[33665]: pgmap v922: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:47.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:47 smithi045 ceph-mon[30165]: pgmap v922: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:47.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:47 smithi072 ceph-mon[33912]: pgmap v922: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:49.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:49 smithi092 ceph-mon[33665]: pgmap v923: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:49.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:49 smithi045 ceph-mon[30165]: pgmap v923: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:49.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:49 smithi072 ceph-mon[33912]: pgmap v923: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:51.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:51 smithi092 ceph-mon[33665]: pgmap v924: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:51.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:51 smithi045 ceph-mon[30165]: pgmap v924: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:51.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:51 smithi072 ceph-mon[33912]: pgmap v924: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:53 smithi092 ceph-mon[33665]: pgmap v925: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:53.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:53 smithi045 ceph-mon[30165]: pgmap v925: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:53.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:53 smithi072 ceph-mon[33912]: pgmap v925: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:55.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:55 smithi092 ceph-mon[33665]: pgmap v926: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:55.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:55 smithi045 ceph-mon[30165]: pgmap v926: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:55.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:55 smithi072 ceph-mon[33912]: pgmap v926: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:57 smithi092 ceph-mon[33665]: pgmap v927: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:57.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:57 smithi045 ceph-mon[30165]: pgmap v927: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:57.665 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:57 smithi072 ceph-mon[33912]: pgmap v927: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:42:59.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:42:59 smithi072 ceph-mon[33912]: pgmap v928: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:59.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:42:59 smithi092 ceph-mon[33665]: pgmap v928: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:42:59.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:42:59 smithi045 ceph-mon[30165]: pgmap v928: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:01.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:01 smithi072 ceph-mon[33912]: pgmap v929: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:01.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:01 smithi092 ceph-mon[33665]: pgmap v929: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:01.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:01 smithi045 ceph-mon[30165]: pgmap v929: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:03 smithi072 ceph-mon[33912]: pgmap v930: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:43:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:43:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:43:03.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:03 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:43:03.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:03 smithi092 ceph-mon[33665]: pgmap v930: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:43:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:43:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:43:03.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:03 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:43:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:03 smithi045 ceph-mon[30165]: pgmap v930: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:43:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:43:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:43:03.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:03 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:43:04.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:04 smithi072 ceph-mon[33912]: pgmap v931: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 1 op/s 2024-05-21T12:43:04.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:04 smithi072 ceph-mon[33912]: pgmap v932: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:04 smithi092 ceph-mon[33665]: pgmap v931: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 1 op/s 2024-05-21T12:43:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:04 smithi092 ceph-mon[33665]: pgmap v932: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:04 smithi045 ceph-mon[30165]: pgmap v931: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 17 KiB/s rd, 1 op/s 2024-05-21T12:43:04.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:04 smithi045 ceph-mon[30165]: pgmap v932: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:06.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:06 smithi072 ceph-mon[33912]: pgmap v933: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:06.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:06 smithi092 ceph-mon[33665]: pgmap v933: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:06.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:06 smithi045 ceph-mon[30165]: pgmap v933: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:08.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:08 smithi072 ceph-mon[33912]: pgmap v934: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:08.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:08 smithi092 ceph-mon[33665]: pgmap v934: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:08.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:08 smithi045 ceph-mon[30165]: pgmap v934: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:10.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:10 smithi072 ceph-mon[33912]: pgmap v935: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:10.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:10 smithi092 ceph-mon[33665]: pgmap v935: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:10.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:10 smithi045 ceph-mon[30165]: pgmap v935: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 20 KiB/s rd, 1 op/s 2024-05-21T12:43:12.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:12 smithi072 ceph-mon[33912]: pgmap v936: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 10 KiB/s rd, 0 op/s 2024-05-21T12:43:12.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:12 smithi092 ceph-mon[33665]: pgmap v936: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 10 KiB/s rd, 0 op/s 2024-05-21T12:43:12.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:12 smithi045 ceph-mon[30165]: pgmap v936: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 10 KiB/s rd, 0 op/s 2024-05-21T12:43:14.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:14 smithi072 ceph-mon[33912]: pgmap v937: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:43:14.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:14 smithi092 ceph-mon[33665]: pgmap v937: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:43:14.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:14 smithi045 ceph-mon[30165]: pgmap v937: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 18 KiB/s rd, 1 op/s 2024-05-21T12:43:16.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:16 smithi072 ceph-mon[33912]: pgmap v938: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:16.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:16 smithi092 ceph-mon[33665]: pgmap v938: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:16.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:16 smithi045 ceph-mon[30165]: pgmap v938: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:18.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:18 smithi072 ceph-mon[33912]: pgmap v939: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:18.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:18 smithi092 ceph-mon[33665]: pgmap v939: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:18.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:18 smithi045 ceph-mon[30165]: pgmap v939: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:20.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:20 smithi072 ceph-mon[33912]: pgmap v940: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:20.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:20 smithi092 ceph-mon[33665]: pgmap v940: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:20.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:20 smithi045 ceph-mon[30165]: pgmap v940: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:22.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:22 smithi072 ceph-mon[33912]: pgmap v941: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:22.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:22 smithi092 ceph-mon[33665]: pgmap v941: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:22.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:22 smithi045 ceph-mon[30165]: pgmap v941: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:24.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:24 smithi072 ceph-mon[33912]: pgmap v942: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:24.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:24 smithi092 ceph-mon[33665]: pgmap v942: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:24.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:24 smithi045 ceph-mon[30165]: pgmap v942: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:26.368 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:26 smithi045 ceph-mon[30165]: pgmap v943: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:26.418 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:26 smithi072 ceph-mon[33912]: pgmap v943: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:26.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:26 smithi092 ceph-mon[33665]: pgmap v943: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:28.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:28 smithi072 ceph-mon[33912]: pgmap v944: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:28.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:28 smithi092 ceph-mon[33665]: pgmap v944: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:28.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:28 smithi045 ceph-mon[30165]: pgmap v944: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:30.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:30 smithi072 ceph-mon[33912]: pgmap v945: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:30.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:30 smithi092 ceph-mon[33665]: pgmap v945: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:30.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:30 smithi045 ceph-mon[30165]: pgmap v945: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:32.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:32 smithi072 ceph-mon[33912]: pgmap v946: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:32.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:32 smithi092 ceph-mon[33665]: pgmap v946: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:32.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:32 smithi045 ceph-mon[30165]: pgmap v946: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:34.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:34 smithi072 ceph-mon[33912]: pgmap v947: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:34.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:34 smithi092 ceph-mon[33665]: pgmap v947: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:34.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:34 smithi045 ceph-mon[30165]: pgmap v947: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:36.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:36 smithi072 ceph-mon[33912]: pgmap v948: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:36.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:36 smithi092 ceph-mon[33665]: pgmap v948: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:36.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:36 smithi045 ceph-mon[30165]: pgmap v948: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:37.043 INFO:tasks.nvmeof.[nvmeof.thrasher]:done waiting before reviving 2024-05-21T12:43:37.043 INFO:tasks.nvmeof.[nvmeof.thrasher]:display and verify stats: 2024-05-21T12:43:37.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:43:37.075 INFO:teuthology.orchestra.run.smithi045.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service - Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:43:37.075 INFO:teuthology.orchestra.run.smithi045.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:43:37.075 INFO:teuthology.orchestra.run.smithi045.stdout: Active: active (running) since Tue 2024-05-21 12:39:51 UTC; 3min 45s ago 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76953 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76954 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: Main PID: 77017 (conmon) 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: Tasks: 126 (limit: 202854) 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: Memory: 155.3M 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: CPU: 3min 48.207s 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: ├─libpod-payload-40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77019 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77021 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:43:37.076 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77039 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.a --gateway-address 172.21.15.45:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.45:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.a -k /etc/ceph/keyring 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77090 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout: │ └─77886 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout: └─runtime 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout: └─77017 /usr/bin/conmon --api-version 1 -c 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f -u 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata -p /run/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:43:37.077 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: OPTIMIZED 2024-05-21T12:43:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:43:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: ], , client address: IPv4 172.21.15.45:46018 2024-05-21T12:43:37.078 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:41:49 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [2024-05-21 12:41:49.136102] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000236 seconds 2024-05-21T12:43:37.082 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:43:37.112 INFO:teuthology.orchestra.run.smithi072.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service - Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:43:37.112 INFO:teuthology.orchestra.run.smithi072.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:43:37.112 INFO:teuthology.orchestra.run.smithi072.stdout: Active: active (running) since Tue 2024-05-21 12:36:12 UTC; 7min ago 2024-05-21T12:43:37.112 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56908 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.112 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56909 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.112 INFO:teuthology.orchestra.run.smithi072.stdout: Main PID: 56971 (conmon) 2024-05-21T12:43:37.113 INFO:teuthology.orchestra.run.smithi072.stdout: Tasks: 126 (limit: 202814) 2024-05-21T12:43:37.113 INFO:teuthology.orchestra.run.smithi072.stdout: Memory: 164.8M 2024-05-21T12:43:37.113 INFO:teuthology.orchestra.run.smithi072.stdout: CPU: 7min 34.453s 2024-05-21T12:43:37.113 INFO:teuthology.orchestra.run.smithi072.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:43:37.113 INFO:teuthology.orchestra.run.smithi072.stdout: ├─libpod-payload-8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:43:37.113 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56973 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:43:37.113 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56975 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:43:37.114 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56993 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.b --gateway-address 172.21.15.72:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.72:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.b -k /etc/ceph/keyring 2024-05-21T12:43:37.114 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─57044 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:43:37.114 INFO:teuthology.orchestra.run.smithi072.stdout: │ └─57834 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:43:37.114 INFO:teuthology.orchestra.run.smithi072.stdout: └─runtime 2024-05-21T12:43:37.114 INFO:teuthology.orchestra.run.smithi072.stdout: └─56971 /usr/bin/conmon --api-version 1 -c 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -u 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata -p /run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:43:37.114 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:43:37.114 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:43:37.115 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:43:37.115 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:43:37.115 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:43:37.115 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:43:37.115 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:43:37.115 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:43:37.115 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:43:37.115 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:43:37.116 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:51082 2024-05-21T12:43:37.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout:× ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service - Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Active: failed (Result: exit-code) since Tue 2024-05-21 12:41:35 UTC; 2min 2s ago 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Duration: 1min 31.602s 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 58193 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 58194 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 59711 ExecStop=/bin/bash -c bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.stop (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 59794 ExecStopPost=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.poststop (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 59825 ExecStopPost=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:43:37.155 INFO:teuthology.orchestra.run.smithi092.stdout: Main PID: 58253 (code=exited, status=137) 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: CPU: 1min 44.666s 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:24 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[58253]: [21-May-2024 12:41:24] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log.gz 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:34 smithi092 bash[59739]: time="2024-05-21T12:41:34Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c in 10 seconds, resorting to SIGKILL" 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:34 smithi092 podman[59739]: 2024-05-21 12:41:34.242162903 +0000 UTC m=+10.137691270 container died 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, build-date=2024-05-19 17:57:19 UTC, io.ceph.git.branch=tags/1.2.9, org.centos.version=9.0-21.el9, io.spdk.build.ceph-release=19.0.0, io.ceph.version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, distribution-scope=public, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, name=ceph-nvmeof, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.git.repo=https://github.com/ceph/spdk.git, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, url=https://github.com/ceph/ceph-nvmeof, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., com.redhat.component=ubi9-container, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.version=24.01, summary=Ceph NVMe over Fabrics Gateway, maintainer=Ceph Developers , vendor=Red Hat, Inc., io.ceph.nvmeof=None, version=1.2.9, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.component=ceph-nvmeof, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.branch=undefined, io.ceph.maintainer=Ceph Developers , io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., release=, io.spdk.build.makeflags=, io.buildah.version=1.29.0, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.pkgdep-args=--rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.openshift.wants=ceph,rbd, architecture=x86_64, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF).) 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:34 smithi092 podman[59739]: 2024-05-21 12:41:34.624894596 +0000 UTC m=+10.520422955 container cleanup 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.ceph-release=19.0.0, name=ceph-nvmeof, build-date=2024-05-19 17:57:19 UTC, io.buildah.version=1.29.0, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., com.redhat.component=ubi9-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, distribution-scope=public, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.version=24.01, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.openshift.wants=ceph,rbd, vendor=Red Hat, Inc., io.ceph.maintainer=Ceph Developers , release=, io.spdk.build.makeflags=, vcs-type=git, org.centos.version=9.0-21.el9, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.branch=undefined, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, maintainer=Ceph Developers , description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.component=ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, url=https://github.com/ceph/ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.nvmeof=None, version=1.2.9, architecture=x86_64, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.summary=Ceph NVMe over Fabrics Gateway) 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:34 smithi092 bash[59739]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:34 smithi092 podman[59779]: 2024-05-21 12:41:34.859592029 +0000 UTC m=+0.616853058 container remove 61ef664411b6f0c871620e1455fc3238401d90efe12ce53efc2b9bd689d5c615 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.openshift.wants=ceph,rbd, maintainer=Ceph Developers , com.redhat.component=ubi9-container, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.git.branch=undefined, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.ceph-release=19.0.0, io.ceph.version=1.2.9, io.spdk.version=24.01, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., distribution-scope=public, org.centos.version=9.0-21.el9, io.spdk.build.makeflags=, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, io.spdk.build.target-arch=, io.ceph.component=ceph-nvmeof, io.ceph.maintainer=Ceph Developers , io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.repo=https://github.com/ceph/spdk.git, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, summary=Ceph NVMe over Fabrics Gateway, io.ceph.nvmeof=None, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, vcs-type=git, io.spdk.build.pkgdep-args=--rbd, name=ceph-nvmeof, build-date=2024-05-19 17:57:19 UTC, version=1.2.9, io.buildah.version=1.29.0, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, url=https://github.com/ceph/ceph-nvmeof, vendor=Red Hat, Inc., architecture=x86_64, release=, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec) 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:34 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:35 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Failed with result 'exit-code'. 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:35 smithi092 systemd[1]: Stopped Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:43:37.156 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:41:35 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Consumed 1min 44.666s CPU time. 2024-05-21T12:43:37.157 DEBUG:teuthology.orchestra.run:got remote process result: 3 2024-05-21T12:43:37.158 DEBUG:teuthology.orchestra.run.smithi137:> ceph orch ls && ceph orch ps --daemon-type nvmeof && ceph health detail && ceph -s && sudo nvme list-subsys /dev/nvme5n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n20 | grep 'live optimized' 2024-05-21T12:43:37.459 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T12:43:37.459 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 1/1 8m ago 24m smithi045=x;count:1 2024-05-21T12:43:37.460 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 8m ago 25m smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:43:37.460 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.mypool ?:4420,5500,8009 1/3 8m ago 19m smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:43:37.460 INFO:teuthology.orchestra.run.smithi137.stdout:osd 8 8m ago - 2024-05-21T12:43:37.747 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:43:37.747 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.a smithi045 *:5500,4420,8009 error 8m ago 19m - - 2024-05-21T12:43:37.747 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.b smithi072 *:5500,4420,8009 error 8m ago 19m - - 2024-05-21T12:43:37.747 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.c smithi092 *:5500,4420,8009 running (19m) 8m ago 19m 160M - 1.2.9 30dd525ace30 5e5a8ffd2ba6 2024-05-21T12:43:38.095 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:43:38.096 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:43:38.096 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:43:38.096 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:43:38.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:38 smithi072 ceph-mon[33912]: pgmap v949: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:38.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:38 smithi072 ceph-mon[33912]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:43:38.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:38 smithi072 ceph-mon[33912]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:43:38.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:38 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/651371472' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:43:38.439 INFO:teuthology.orchestra.run.smithi137.stdout: cluster: 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: health: HEALTH_WARN 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: 2 failed cephadm daemon(s) 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: services: 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: mon: 3 daemons, quorum a,c,b (age 25m) 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: mgr: x(active, since 26m) 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: osd: 8 osds: 8 up (since 21m), 8 in (since 21m) 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: data: 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: pools: 2 pools, 33 pgs 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: objects: 485 objects, 1.0 GiB 2024-05-21T12:43:38.440 INFO:teuthology.orchestra.run.smithi137.stdout: usage: 4.3 GiB used, 711 GiB / 715 GiB avail 2024-05-21T12:43:38.441 INFO:teuthology.orchestra.run.smithi137.stdout: pgs: 33 active+clean 2024-05-21T12:43:38.441 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:43:38.441 INFO:teuthology.orchestra.run.smithi137.stdout: io: 2024-05-21T12:43:38.441 INFO:teuthology.orchestra.run.smithi137.stdout: client: 23 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-05-21T12:43:38.441 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:43:38.470 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.488 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:38 smithi092 ceph-mon[33665]: pgmap v949: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:38.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:38 smithi092 ceph-mon[33665]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:43:38.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:38 smithi092 ceph-mon[33665]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:43:38.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:38 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/651371472' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:43:38.506 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.525 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.543 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.561 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:38 smithi045 ceph-mon[30165]: pgmap v949: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:43:38.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:38 smithi045 ceph-mon[30165]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:43:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:38 smithi045 ceph-mon[30165]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:43:38.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:38 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/651371472' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:43:38.579 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.597 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.616 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.634 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.652 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.670 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.688 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.707 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.725 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.743 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.761 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.779 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.797 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.816 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.834 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.852 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.871 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.889 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.908 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.926 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.944 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.962 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.981 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:38.999 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.017 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.036 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.054 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.072 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.090 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.108 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.127 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.145 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.163 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.181 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.200 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.218 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.236 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.255 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.273 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.291 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.309 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.346 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.365 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.383 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.401 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:39 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/1042267553' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:43:39.419 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.438 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.457 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.475 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.493 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:39 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/1042267553' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:43:39.512 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.531 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.550 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:43:39.550 INFO:tasks.nvmeof.[nvmeof.thrasher]:reviving nvmeof.c 2024-05-21T12:43:39.550 INFO:tasks.nvmeof.ceph.nvmeof.nvmeof.c:Restarting nvmeof.nvmeof.c (starting--it wasn't running)... 2024-05-21T12:43:39.550 DEBUG:teuthology.orchestra.run.smithi092:nvmeof.nvmeof.c> sudo journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:43:39.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl start ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:43:39.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:39 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/1042267553' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:43:39.856 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:39 smithi092 systemd[1]: Starting Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:43:40.132 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:39 smithi092 podman[60217]: 2024-05-21 12:43:39.758947033 +0000 UTC m=+0.017593054 image pull 30dd525ace30bc703cdf4a290b3be2b24f6ca7b74757424a10799eaf370636b4 quay.io/ceph/nvmeof:1.2 2024-05-21T12:43:40.133 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:39 smithi092 podman[60217]: 2024-05-21 12:43:39.869359017 +0000 UTC m=+0.128005021 container create 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, version=1.2.9, release=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., org.centos.version=9.0-21.el9, summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.target-arch=, io.ceph.git.branch=tags/1.2.9, com.redhat.component=ubi9-container, distribution-scope=public, io.ceph.url=https://github.com/ceph/ceph-nvmeof, architecture=x86_64, vendor=Red Hat, Inc., io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.version=1.2.9, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.build.makeflags=, io.openshift.wants=ceph,rbd, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, build-date=2024-05-19 17:57:19 UTC, io.ceph.nvmeof=None, name=ceph-nvmeof, io.ceph.maintainer=Ceph Developers , io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.buildah.version=1.29.0, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.ceph-release=19.0.0, vcs-type=git, io.ceph.component=ceph-nvmeof, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.version=24.01, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.branch=undefined, maintainer=Ceph Developers , url=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:43:40.133 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 podman[60217]: 2024-05-21 12:43:40.03816072 +0000 UTC m=+0.296806743 container init 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.buildah.version=1.29.0, io.spdk.build.pkgdep-args=--rbd, io.openshift.wants=ceph,rbd, vcs-type=git, io.ceph.nvmeof=None, io.spdk.build.makeflags=, url=https://github.com/ceph/ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, version=1.2.9, io.ceph.component=ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, release=, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, build-date=2024-05-19 17:57:19 UTC, io.ceph.url=https://github.com/ceph/ceph-nvmeof, summary=Ceph NVMe over Fabrics Gateway, com.redhat.component=ubi9-container, io.spdk.build.ceph-release=19.0.0, distribution-scope=public, name=ceph-nvmeof, vendor=Red Hat, Inc., architecture=x86_64, io.ceph.git.branch=tags/1.2.9, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.target-arch=, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.git.branch=undefined, io.ceph.version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.version=24.01, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, org.centos.version=9.0-21.el9, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.maintainer=Ceph Developers ) 2024-05-21T12:43:40.133 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 podman[60217]: 2024-05-21 12:43:40.040919858 +0000 UTC m=+0.299565872 container start 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.ceph.nvmeof=None, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.spdk.version=24.01, io.spdk.build.ceph-release=19.0.0, io.ceph.maintainer=Ceph Developers , distribution-scope=public, io.spdk.build.pkgdep-args=--rbd, url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, io.ceph.git.branch=tags/1.2.9, io.spdk.build.makeflags=, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, org.centos.version=9.0-21.el9, io.buildah.version=1.29.0, io.ceph.url=https://github.com/ceph/ceph-nvmeof, vendor=Red Hat, Inc., build-date=2024-05-19 17:57:19 UTC, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, maintainer=Ceph Developers , architecture=x86_64, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.repo=https://github.com/ceph/spdk.git, name=ceph-nvmeof, vcs-type=git, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, version=1.2.9, io.ceph.component=ceph-nvmeof, release=, io.ceph.summary=Ceph NVMe over Fabrics Gateway, summary=Ceph NVMe over Fabrics Gateway, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.branch=undefined, io.ceph.version=1.2.9, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.wants=ceph,rbd, com.redhat.component=ubi9-container) 2024-05-21T12:43:40.144 INFO:tasks.nvmeof.[nvmeof.thrasher]:waiting for 76 secs before thrashing 2024-05-21T12:43:40.384 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-mon[33665]: pgmap v950: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 bash[60217]: 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 systemd[1]: Started Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO utils.py:258: Initialize gateway log level to "INFO" 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO utils.py:271: Log files will be saved in /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c, using rotation 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:78: Using NVMeoF gateway version 1.2.9 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:81: Using SPDK version 24.01 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:84: Using vstart cluster version based on 19.0.0 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:87: NVMeoF gateway built on: 2024-05-19 17:57:19 UTC 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:90: NVMeoF gateway Git repository: https://github.com/ceph/ceph-nvmeof 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:93: NVMeoF gateway Git branch: tags/1.2.9 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:96: NVMeoF gateway Git commit: 06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec 2024-05-21T12:43:40.385 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:102: SPDK Git repository: https://github.com/ceph/spdk.git 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:105: SPDK Git branch: undefined 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:108: SPDK Git commit: dade5b4c96a647e63ef96d9a5e05f3b64ea0598a 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:59: Using configuration file /src/ceph-nvmeof.conf 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:61: ====================================== Configuration file content ====================================== 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: # This file is generated by cephadm. 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: [gateway] 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: name = client.nvmeof.nvmeof.c 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: group = 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: addr = 172.21.15.92 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: port = 5500 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: enable_auth = False 2024-05-21T12:43:40.386 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: state_update_notify = True 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: state_update_interval_sec = 5 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: enable_spdk_discovery_controller = False 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: enable_prometheus_exporter = True 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: prometheus_exporter_ssl = False 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: prometheus_port = 10008 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: verify_nqns = True 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: omap_file_lock_duration = 20 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: omap_file_lock_retries = 30 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: omap_file_lock_retry_sleep_interval = 1.0 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: omap_file_update_reloads = 10 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: allowed_consecutive_spdk_ping_failures = 1 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: spdk_ping_interval_in_seconds = 2.0 2024-05-21T12:43:40.387 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: ping_spdk_under_lock = False 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: enable_monitor_client = True 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: [gateway-logs] 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: log_level = INFO 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: log_files_enabled = True 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: log_files_rotation_enabled = True 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: verbose_log_messages = True 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: max_log_file_size_in_mb = 10 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: max_log_files_count = 20 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: max_log_directory_backups = 10 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: log_directory = /var/log/ceph/ 2024-05-21T12:43:40.388 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: [discovery] 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: addr = 172.21.15.92 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: port = 8009 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: [ceph] 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: pool = mypool 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: config_file = /etc/ceph/ceph.conf 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: id = nvmeof.nvmeof.c 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: [mtls] 2024-05-21T12:43:40.392 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: server_key = ./server.key 2024-05-21T12:43:40.393 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: client_key = ./client.key 2024-05-21T12:43:40.393 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: server_cert = ./server.crt 2024-05-21T12:43:40.393 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: client_cert = ./client.crt 2024-05-21T12:43:40.393 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: 2024-05-21T12:43:40.393 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: [spdk] 2024-05-21T12:43:40.393 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: tgt_path = /usr/local/bin/nvmf_tgt 2024-05-21T12:43:40.393 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: rpc_socket_dir = /var/tmp/ 2024-05-21T12:43:40.393 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: rpc_socket_name = spdk.sock 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: timeout = 60.0 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: bdevs_per_cluster = 32 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: log_level = WARNING 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: conn_retries = 10 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: transports = tcp 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: transport_tcp_options = {"in_capsule_data_size": 8192, "max_io_qpairs_per_ctrlr": 7} 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: [monitor] 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:65: timeout = 1.0 2024-05-21T12:43:40.394 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO config.py:66: ======================================================================================================== 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:90: Starting gateway client.nvmeof.nvmeof.c 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:157: Starting serve, monitor client version: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:236: Starting /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.c --gateway-address 172.21.15.92:5500 --gateway-pool mypool --gateway-group --monitor-group-address 172.21.15.92:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.c -k /etc/ceph/keyring 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:240: monitor client process id: 19 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:146: MonitorGroup server is listening on 172.21.15.92:5499 for group id 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.365+0000 7fbfb9f415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.365+0000 7fbfb9f415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() gateway name: client.nvmeof.nvmeof.c pool:mypool group: address: 172.21.15.92:5500 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.366+0000 7fbfb9f415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() nvmeof Registered monc callback 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.368+0000 7fbfb9f415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() monc.authentication done 2024-05-21T12:43:40.395 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.368+0000 7fbfb9f415c0 0 nvmeofgw void NVMeofGwMonitorClient::tick() 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.369+0000 7fbfb9f415c0 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.369+0000 7fbfb9f415c0 0 nvmeofgw void NVMeofGwMonitorClient::send_beacon() sending beacon as gid 15558 availability 0 osdmap_epoch 0 gwmap_epoch 0 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.369+0000 7fbfb9f415c0 0 nvmeofgw int NVMeofGwMonitorClient::init() Complete. 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.369+0000 7fbfb5ede640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 2048 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.369+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) handle nvmeof gw map: {mypool,=NvmeGwMap 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.a -> NvmeGwState { group id: 0 gw_map_epoch 144 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [OPTIMIZED ,0,INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [OPTIMIZED ,0,INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [OPTIMIZED ,0,INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.b -> NvmeGwState { group id: 1 gw_map_epoch 144 availablilty AVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,OPTIMIZED ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }} 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: nvmeofgw { == gw_id: client.nvmeof.nvmeof.c -> NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] }}}} 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.369+0000 7fbfb5ede640 0 nvmeofgw bool get_gw_state(const char*, const std::map, std::__cxx11::basic_string >, std::map, NvmeGwState> >&, const NvmeGroupKey&, const NvmeGwId&, NvmeGwState&) can not find group (mypool,) old map map: {} 2024-05-21T12:43:40.396 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.369+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_group_id: 2 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:135: Gateway client.nvmeof.nvmeof.c group id=2 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:149: Stopping the MonitorGroup server... 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.397 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.398 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.399 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 1 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 2 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 3 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 4 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 5 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 6 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.400 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 7 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 8 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 9 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 10 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 11 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 12 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 13 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 14 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 15 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) grpid 16 state: NvmeGwState { group id: 2 gw_map_epoch 144 availablilty UNAVAILABLE GwSubsystems: [ NqnState( nqn: nqn.2016-06.io.spdk:cnode1, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode2, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) NqnState( nqn: nqn.2016-06.io.spdk:cnode3, [INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0,INACCESSIBLE ,0] ) ] } 2024-05-21T12:43:40.401 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.371+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.373+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.374+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:152: The MonitorGroup gRPC server stopped... 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:349: SPDK Target Path: /usr/local/bin/nvmf_tgt 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:351: SPDK Socket: /var/tmp/spdk.sock 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:357: Starting /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.376+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.377+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:370: SPDK process id: 70 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:371: Attempting to initialize SPDK: rpc_socket: /var/tmp/spdk.sock, conn_retries: 300, timeout: 60.0 2024-05-21T12:43:40.402 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: INFO: Setting log level to WARNING 2024-05-21T12:43:40.403 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO client.py:110: Setting log level to WARNING 2024-05-21T12:43:40.403 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.379+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.403 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.380+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.403 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.382+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.403 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.383+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.403 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.385+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.403 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.386+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.403 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.388+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.415 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:40 smithi072 ceph-mon[33912]: pgmap v950: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:40.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:40 smithi045 ceph-mon[30165]: pgmap v950: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:40.635 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.389+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.635 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.392688] Starting SPDK v24.01 / DPDK 23.11.0 initialization... 2024-05-21T12:43:40.635 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.392755] [ DPDK EAL parameters: nvmf --no-shconf -c 0x1 --no-pci --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --iova-mode=pa --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid70 ] 2024-05-21T12:43:40.635 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.391+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.635 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.393+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.635 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.394+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.635 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.395+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.635 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.397+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.636 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.398+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.636 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.400+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.636 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.401+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.636 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.403+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.636 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.404+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.636 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.406+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.636 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.407+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.636 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.409+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.637 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.410+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.637 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.412+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.637 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.413+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.637 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.415+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.637 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.416+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.637 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.418+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.637 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.419+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.637 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.421+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.422+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.424+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.425+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.427+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.429+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.430+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.432+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.433+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.435+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.437+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.438+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.638 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.440+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.441+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.443+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.445+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.446+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.448+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.449+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.451+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.453+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.454+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.456+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.639 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.457+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.459+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.461+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.462+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.464+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.466+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.467+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.469+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.471+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.472+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.474+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.475+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.640 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.477+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.641 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.478+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.641 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.480+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.641 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.481+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.641 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.483+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.641 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.484+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.641 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.486+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.642 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.487+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.489+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.490+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.492+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.493+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.495+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.496+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.498+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.499+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.501+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.502+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.503+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.643 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.505+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.506900] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.507+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.509+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.510+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.512+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.514+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.515+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.517+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.518+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.520+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.521+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.523+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.644 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.525+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.526+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.528+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.530+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.532+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.533+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.535+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.536+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.538+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.539+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.541+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.542+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.543+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.645 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.545+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.546+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.548+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.549+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.551+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.552+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.554+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.555+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.557+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.559+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.560+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.562687] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.562+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.646 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.563+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.565+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.567+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.569+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.571+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.572+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.574+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.575+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.577+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.579+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.580+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.581+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.583+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.585+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.647 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.587+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.589+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.591+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.593+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.595+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.597+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.599+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.601+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.603+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.605+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.606+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.608+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.610+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.648 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.611+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.613+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.615+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.616+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.618+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.619+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.621+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.623+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.624+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.626+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.628+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.629+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.631+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.633+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.649 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.634+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.650 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.636+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.650 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.637+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.650 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.639+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.898 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.640+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.898 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.642+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.643+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.645+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.646+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.648+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.649+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.650+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.652+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.653+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.655+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.656+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.658+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.899 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.659+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.661+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.662+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.663+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.665+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.666+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.668+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.669+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.671+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.672+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.674+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.676+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.677+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.900 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.679+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.901 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.680+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.901 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.682+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.901 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.683+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.901 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.685+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.901 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.686+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.901 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.688+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.902 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.689+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.902 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.691+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.902 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.692+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.902 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.694+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.695+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.696+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.698+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.699+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.701+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.702+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.704+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.705+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.707+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.708+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.709+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.903 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.711+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.712+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.714+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.715+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.717+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.718+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.720+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.721+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.722+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.724+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.725+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.904 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.727+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.728+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.730+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.731+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.732+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.734+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.735+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.737+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.738+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.740+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.741+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.742+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.744+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.905 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.745+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.747+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.748+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.750+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.751+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.753+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.754+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.755+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.757+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.758+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.760+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.761+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.763+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.906 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.764+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.765+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.767+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.768+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.770+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.771+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.773+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.774+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.775+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.777+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.778+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.907 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.780+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.781912] tcp.c: 659:nvmf_tcp_create: *NOTICE*: *** TCP Transport Init *** 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.781+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.783+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.784+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.786+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.787+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.788+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.790+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.791+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.793+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:268: Discovery service process id: 885 2024-05-21T12:43:40.908 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:264: Starting ceph nvmeof discovery service 2024-05-21T12:43:40.909 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.794+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.909 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.796+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.909 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.797+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.909 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.799+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.909 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.801+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.909 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.802+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.909 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.804+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.910 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.806+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.910 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO cephutils.py:74: Connected to Ceph with version "19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev)" 2024-05-21T12:43:40.910 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:86: Requested huge pages count is 2048 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:105: Actual huge pages count is 4096 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:190: NVMeoF bdevs per cluster: 32 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.808+0000 7fbfb5ede640 0 nvmeofgw void NVMeofGwMonitorClient::handle_nvmeof_gw_map(ceph::ref_t) GRPC set_ana_state failed 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO state.py:355: nvmeof.state OMAP object already exists. 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO discovery.py:322: log pages info from omap: nvmeof.state 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode2, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO server.py:195: Prometheus endpoint is enabled 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO prometheus.py:98: Prometheus exporter running in http mode, listening on port 10008 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 1 2024-05-21T12:43:40.911 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 2 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 3 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 4 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.912 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 5 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 6 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 7 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.913 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 8 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 9 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 10 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.914 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 11 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 12 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 13 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 14 2024-05-21T12:43:40.915 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 15 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 16 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 1 2024-05-21T12:43:40.916 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 2 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 3 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 4 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.917 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 5 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 6 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 7 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.918 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 8 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 9 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 10 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 11 2024-05-21T12:43:40.919 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 12 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 13 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 14 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.920 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 15 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 16 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 1 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.921 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 2 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 3 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 4 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 5 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 6 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 7 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 8 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 9 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 10 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 11 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 12 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 13 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 14 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 15 2024-05-21T12:43:40.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 16 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: INACCESSIBLE 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: ], , client address: IPv4 172.21.15.92:59428 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode1 not found 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] WARNING grpc.py:2079: Subsystem nqn.2016-06.io.spdk:cnode3 not found 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:2648: Received request to get gateway's info 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO prometheus.py:131: Stats for all bdevs will be provided 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode1, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.822+0000 7fbfb5ede640 0 ms_deliver_dispatch: unhandled message 0x557e349a3180 nvmeofgwmap magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:43:40.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.822+0000 7fbfb5ede640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 4 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.822+0000 7fbfb5ede640 0 ms_deliver_dispatch: unhandled message 0x557e34993d40 mon_map magic: 0 from mon.0 v2:172.21.15.45:3300/0 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2024-05-21T12:43:40.823+0000 7fbfb5ede640 0 nvmeofgw virtual bool NVMeofGwMonitorClient::ms_dispatch2(ceph::ref_t&) got map type 41 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO discovery.py:330: discovery addr: 172.21.15.92 port: 8009 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:507: Received request to create subsystem nqn.2016-06.io.spdk:cnode3, enable_ha: True, max_namespaces: 256, context: None 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.865882] nvmf_rpc.c: 275:rpc_nvmf_get_subsystems: *WARNING*: rpc_nvmf_get_subsystems: deprecated feature listener.transport is deprecated in favor of trtype to be removed in v24.05 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:274: Received request to create bdev bdev_b87a5ac5-b293-4485-b785-2a12b9a685af from mypool/myimage8 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.882022] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_3_0, nonce 172.21.15.92:0/2630501894, rc 0 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.882096] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_3_0, nonce 172.21.15.92:0/2630501894 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:250: Allocated cluster name='cluster_context_3_0' nonce='172.21.15.92:0/2630501894' anagrp=3 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 1 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.897902] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af rbd disk to lun 2024-05-21T12:43:40.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:760: Received request to add bdev_b87a5ac5-b293-4485-b785-2a12b9a685af to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 8 and UUID b87a5ac5-b293-4485-b785-2a12b9a685af 2024-05-21T12:43:41.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-mon[33665]: from='client.? 172.21.15.92:0/3926394864' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:43:41.156 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:274: Received request to create bdev bdev_e349eac9-5fb1-4829-8523-49a423678213 from mypool/myimage6 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.921957] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_1_0, nonce 172.21.15.92:0/1947385835, rc 0 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.922052] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_1_0, nonce 172.21.15.92:0/1947385835 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:250: Allocated cluster name='cluster_context_1_0' nonce='172.21.15.92:0/1947385835' anagrp=1 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 1 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:40.936266] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e349eac9-5fb1-4829-8523-49a423678213 rbd disk to lun 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:40] INFO grpc.py:760: Received request to add bdev_e349eac9-5fb1-4829-8523-49a423678213 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 6 and UUID e349eac9-5fb1-4829-8523-49a423678213 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:43:41.157 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 from mypool/myimage37 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.045481] bdev_rbd.c:1278:rbd_register_cluster: *NOTICE*: rbd_register_cluster cluster_context_2_0, nonce 172.21.15.92:0/3781527418, rc 0 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.046006] bdev_rbd.c: 968:dump_cluster_nonce: *NOTICE*: dump_cluster_nonce cluster_context_2_0, nonce 172.21.15.92:0/3781527418 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:250: Allocated cluster name='cluster_context_2_0' nonce='172.21.15.92:0/3781527418' anagrp=2 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 1 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.061411] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 rbd disk to lun 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_899d8ccd-f410-47e8-9ef3-478a9d409408 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 17 and UUID 899d8ccd-f410-47e8-9ef3-478a9d409408 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 from mypool/myimage32 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 2 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.083973] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 rbd disk to lun 2024-05-21T12:43:41.158 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_38942ad5-39c0-4396-85fa-87bbd7a3d770 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 12 and UUID 38942ad5-39c0-4396-85fa-87bbd7a3d770 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_d2218777-49ff-4c2d-84de-6d703a754573 from mypool/myimage59 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 3 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.104688] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d2218777-49ff-4c2d-84de-6d703a754573 rbd disk to lun 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_d2218777-49ff-4c2d-84de-6d703a754573 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 19 and UUID d2218777-49ff-4c2d-84de-6d703a754573 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_5aaed34a-1696-499e-b0b0-fad88d740507 from mypool/myimage25 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 2 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.125370] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 rbd disk to lun 2024-05-21T12:43:41.159 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_5aaed34a-1696-499e-b0b0-fad88d740507 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 5 and UUID 5aaed34a-1696-499e-b0b0-fad88d740507 2024-05-21T12:43:41.160 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:43:41.160 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da from mypool/myimage50 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.160 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 4 2024-05-21T12:43:41.160 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.147153] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da rbd disk to lun 2024-05-21T12:43:41.160 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_1e1ea26a-6294-47e4-b3d6-3d11171843da to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 10 and UUID 1e1ea26a-6294-47e4-b3d6-3d11171843da 2024-05-21T12:43:41.160 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:43:41.160 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed from mypool/myimage22 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.160 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 3 2024-05-21T12:43:41.414 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.170047] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed rbd disk to lun 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_b5a82937-0d41-4fa2-8d4d-9f933c80f1ed to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 2 and UUID b5a82937-0d41-4fa2-8d4d-9f933c80f1ed 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b from mypool/myimage24 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 2 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.191171] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b rbd disk to lun 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_d9bfd1c8-81ea-4eb9-8603-5618e621e98b to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 4 and UUID d9bfd1c8-81ea-4eb9-8603-5618e621e98b 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_d826b447-494f-472c-bb8e-d88c0562fa80 from mypool/myimage38 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 5 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.212269] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 rbd disk to lun 2024-05-21T12:43:41.415 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_d826b447-494f-472c-bb8e-d88c0562fa80 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 18 and UUID d826b447-494f-472c-bb8e-d88c0562fa80 2024-05-21T12:43:41.416 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:43:41.416 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a from mypool/myimage18 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.416 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 3 2024-05-21T12:43:41.417 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:41 smithi072 ceph-mon[33912]: from='client.? 172.21.15.92:0/3926394864' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:43:41.417 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.233171] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a rbd disk to lun 2024-05-21T12:43:41.417 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_1af254a8-e8d6-4e20-9188-384ceaa4505a to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 18 and UUID 1af254a8-e8d6-4e20-9188-384ceaa4505a 2024-05-21T12:43:41.417 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:43:41.417 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 from mypool/myimage9 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.417 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 4 2024-05-21T12:43:41.417 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.256723] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 rbd disk to lun 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_867f5e00-8d71-4aae-b31a-dc9fb010b3b6 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 9 and UUID 867f5e00-8d71-4aae-b31a-dc9fb010b3b6 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f from mypool/myimage56 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 6 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.278078] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f rbd disk to lun 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_4a4c30b3-7624-4e49-b965-914eca74bb8f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 16 and UUID 4a4c30b3-7624-4e49-b965-914eca74bb8f 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 from mypool/myimage28 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.418 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 4 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.299053] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 rbd disk to lun 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_6f017e0c-1259-41c2-aa4e-63e3a6d75015 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 8 and UUID 6f017e0c-1259-41c2-aa4e-63e3a6d75015 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea from mypool/myimage29 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 7 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.321329] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea rbd disk to lun 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_deecdc0a-c85f-47e5-a88d-af0d5182e6ea to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 9 and UUID deecdc0a-c85f-47e5-a88d-af0d5182e6ea 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e from mypool/myimage4 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 5 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.342384] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e rbd disk to lun 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_74c74c54-3d4e-4485-ba42-2b584b6fa81e to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 4 and UUID 74c74c54-3d4e-4485-ba42-2b584b6fa81e 2024-05-21T12:43:41.419 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_34a70313-baa0-485f-bd37-d8df76585703 from mypool/myimage53 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 8 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.363423] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_34a70313-baa0-485f-bd37-d8df76585703 rbd disk to lun 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_34a70313-baa0-485f-bd37-d8df76585703 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 13 and UUID 34a70313-baa0-485f-bd37-d8df76585703 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f from mypool/myimage44 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 9 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.384261] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f rbd disk to lun 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_7a09ebd3-49a1-4ee5-8e71-41364492b16f to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 4 and UUID 7a09ebd3-49a1-4ee5-8e71-41364492b16f 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 from mypool/myimage46 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 6 2024-05-21T12:43:41.420 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.405290] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 rbd disk to lun 2024-05-21T12:43:41.421 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 6 and UUID ae876ba4-a37b-4c47-90ea-a61c0ae1dc39 2024-05-21T12:43:41.421 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:43:41.421 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 from mypool/myimage31 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.421 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 7 2024-05-21T12:43:41.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:41 smithi045 ceph-mon[30165]: from='client.? 172.21.15.92:0/3926394864' entity='client.nvmeof.nvmeof.c' cmd={"prefix":"mon versions"} : dispatch 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.428473] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 rbd disk to lun 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 11 and UUID 6fc6c70a-1f65-486d-bdd6-09bf4df6f9a3 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 from mypool/myimage33 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 5 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.449216] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 rbd disk to lun 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_97ec7eca-d2dc-4254-a4ca-12d17cc765b7 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 13 and UUID 97ec7eca-d2dc-4254-a4ca-12d17cc765b7 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 from mypool/myimage41 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.671 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 10 2024-05-21T12:43:41.672 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.473077] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 rbd disk to lun 2024-05-21T12:43:41.672 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_514d58ae-41ac-4a18-bad6-06a6168f1234 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 1 and UUID 514d58ae-41ac-4a18-bad6-06a6168f1234 2024-05-21T12:43:41.672 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:43:41.672 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_346f90e2-ed39-4090-8599-e1283c82d3ad from mypool/myimage15 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.672 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 6 2024-05-21T12:43:41.672 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.494990] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad rbd disk to lun 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_346f90e2-ed39-4090-8599-e1283c82d3ad to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 15 and UUID 346f90e2-ed39-4090-8599-e1283c82d3ad 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 from mypool/myimage5 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 11 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.516181] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 rbd disk to lun 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_2363a1af-0d49-4f75-81f0-8b985c6c90d2 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 5 and UUID 2363a1af-0d49-4f75-81f0-8b985c6c90d2 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 from mypool/myimage30 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 7 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.539454] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 rbd disk to lun 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_42686ed2-6784-4220-a2d9-6fd164e3c2f3 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 10 and UUID 42686ed2-6784-4220-a2d9-6fd164e3c2f3 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:43:41.673 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 from mypool/myimage45 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 8 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.561757] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 rbd disk to lun 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 5 and UUID 2b67ac15-cf2c-4cc6-a1fc-8ee82ca7f327 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_c21863b3-ea38-4777-afc0-6a7011958143 from mypool/myimage26 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 12 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.582938] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c21863b3-ea38-4777-afc0-6a7011958143 rbd disk to lun 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_c21863b3-ea38-4777-afc0-6a7011958143 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 6 and UUID c21863b3-ea38-4777-afc0-6a7011958143 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a from mypool/myimage34 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 8 2024-05-21T12:43:41.674 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.604146] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a rbd disk to lun 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_bd92ec20-9f67-4d5e-aa72-bc5eddab838a to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 14 and UUID bd92ec20-9f67-4d5e-aa72-bc5eddab838a 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd from mypool/myimage51 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 9 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.626108] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd rbd disk to lun 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_ffd15d0b-cb83-4048-beb1-b65319b34ddd to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 11 and UUID ffd15d0b-cb83-4048-beb1-b65319b34ddd 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e from mypool/myimage12 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 10 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.647494] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e rbd disk to lun 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_ee1e99bf-15bd-4039-abfd-8e40153fe06e to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 12 and UUID ee1e99bf-15bd-4039-abfd-8e40153fe06e 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:43:41.675 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_cba00c90-3501-4d43-b5c2-017aac10a02d from mypool/myimage10 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.676 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 9 2024-05-21T12:43:41.676 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.670526] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d rbd disk to lun 2024-05-21T12:43:41.676 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_cba00c90-3501-4d43-b5c2-017aac10a02d to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 10 and UUID cba00c90-3501-4d43-b5c2-017aac10a02d 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 from mypool/myimage14 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 13 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.690689] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 rbd disk to lun 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_71b9a6c7-44e0-4e40-95a2-a76947588015 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 14 and UUID 71b9a6c7-44e0-4e40-95a2-a76947588015 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2, ana group 2 context: None 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_4d993843-ec8c-481b-acef-898a1c83418c from mypool/myimage40 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 10 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.713814] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4d993843-ec8c-481b-acef-898a1c83418c rbd disk to lun 2024-05-21T12:43:41.922 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_4d993843-ec8c-481b-acef-898a1c83418c to nqn.2016-06.io.spdk:cnode2 with ANA group id 2 using NSID 20 and UUID 4d993843-ec8c-481b-acef-898a1c83418c 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_1ac8703e-ed04-493b-ba52-c73153370d7b from mypool/myimage1 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 11 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.734694] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b rbd disk to lun 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_1ac8703e-ed04-493b-ba52-c73153370d7b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 1 and UUID 1ac8703e-ed04-493b-ba52-c73153370d7b 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3, ana group 3 context: None 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_0735e620-3968-43dc-a722-37465b1e9ce6 from mypool/myimage47 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 14 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.757252] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 rbd disk to lun 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_0735e620-3968-43dc-a722-37465b1e9ce6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 3 using NSID 7 and UUID 0735e620-3968-43dc-a722-37465b1e9ce6 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:43:41.923 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_9430b9c3-154d-4bdc-ac40-619f26451491 from mypool/myimage49 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 12 2024-05-21T12:43:41.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.777773] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 rbd disk to lun 2024-05-21T12:43:41.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_9430b9c3-154d-4bdc-ac40-619f26451491 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 9 and UUID 9430b9c3-154d-4bdc-ac40-619f26451491 2024-05-21T12:43:41.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:43:41.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc from mypool/myimage20 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 15 2024-05-21T12:43:41.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.800217] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc rbd disk to lun 2024-05-21T12:43:41.924 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_a6235233-7e8c-4b35-8493-fd5bfbe280cc to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 20 and UUID a6235233-7e8c-4b35-8493-fd5bfbe280cc 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 from mypool/myimage27 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 11 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.823014] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 rbd disk to lun 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_ea2d877e-1a43-4674-a681-a0d8bf464c34 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 7 and UUID ea2d877e-1a43-4674-a681-a0d8bf464c34 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 from mypool/myimage54 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 12 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.843755] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 rbd disk to lun 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 14 and UUID 7b5d8f0c-b651-4f87-a24a-8700ba6f4ee8 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 from mypool/myimage52 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.925 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 13 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.865758] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 rbd disk to lun 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_8bad29f1-fb93-4936-a7b2-067cb9715706 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 12 and UUID 8bad29f1-fb93-4936-a7b2-067cb9715706 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 from mypool/myimage16 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 14 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.888106] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 rbd disk to lun 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_d0f0d22d-353b-42cf-a21e-ff56cdf02783 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 16 and UUID d0f0d22d-353b-42cf-a21e-ff56cdf02783 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 from mypool/myimage39 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 13 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.912570] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 rbd disk to lun 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_d3d6400e-10dc-4ec2-96ec-18e93314b495 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 19 and UUID d3d6400e-10dc-4ec2-96ec-18e93314b495 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:43:41.926 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_e7985387-2968-452b-9709-e37a1a8255ca from mypool/myimage57 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:41.927 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 14 2024-05-21T12:43:42.175 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-mon[33665]: pgmap v951: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:42.175 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.934652] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_e7985387-2968-452b-9709-e37a1a8255ca rbd disk to lun 2024-05-21T12:43:42.175 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_e7985387-2968-452b-9709-e37a1a8255ca to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 17 and UUID e7985387-2968-452b-9709-e37a1a8255ca 2024-05-21T12:43:42.175 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b from mypool/myimage19 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 15 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.956418] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b rbd disk to lun 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_df7d58b1-d12c-4a99-8a5c-768adcfd252b to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 19 and UUID df7d58b1-d12c-4a99-8a5c-768adcfd252b 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_88372a05-54d9-4c17-af42-2b9c02991736 from mypool/myimage21 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 15 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:41.977853] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_88372a05-54d9-4c17-af42-2b9c02991736 rbd disk to lun 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:760: Received request to add bdev_88372a05-54d9-4c17-af42-2b9c02991736 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 1 and UUID 88372a05-54d9-4c17-af42-2b9c02991736 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:274: Received request to create bdev bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b from mypool/myimage2 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.176 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:41] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 16 2024-05-21T12:43:42.177 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.001209] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b rbd disk to lun 2024-05-21T12:43:42.177 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_fd2b5ba2-2f5c-46fb-8916-674d2cbb477b to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 2 and UUID fd2b5ba2-2f5c-46fb-8916-674d2cbb477b 2024-05-21T12:43:42.177 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:43:42.177 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 from mypool/myimage7 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.177 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 16 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.022025] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 rbd disk to lun 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_95c9bca2-e876-4ec5-8958-2e2ca7e60371 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 7 and UUID 95c9bca2-e876-4ec5-8958-2e2ca7e60371 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b from mypool/myimage58 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 17 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.042746] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b rbd disk to lun 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_add34d0b-baf5-4a4e-bfa3-90b32af15b9b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 18 and UUID add34d0b-baf5-4a4e-bfa3-90b32af15b9b 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 from mypool/myimage48 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 16 2024-05-21T12:43:42.178 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.065265] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 rbd disk to lun 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_fbcff93c-1d7a-4216-9960-29e985da87e5 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 8 and UUID fbcff93c-1d7a-4216-9960-29e985da87e5 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_77f81441-23a5-44ee-985c-f37050ecffc6 from mypool/myimage60 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 17 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.087239] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 rbd disk to lun 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_77f81441-23a5-44ee-985c-f37050ecffc6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 20 and UUID 77f81441-23a5-44ee-985c-f37050ecffc6 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1, ana group 1 context: None 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 from mypool/myimage3 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.179 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 18 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.110324] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 rbd disk to lun 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_3185e485-c8a5-4af2-ae8e-15d7100dcdf0 to nqn.2016-06.io.spdk:cnode1 with ANA group id 1 using NSID 3 and UUID 3185e485-c8a5-4af2-ae8e-15d7100dcdf0 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 from mypool/myimage43 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 18 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.132031] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 rbd disk to lun 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_894b8b8c-3043-469c-bfcf-b11efc460de6 to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 3 and UUID 894b8b8c-3043-469c-bfcf-b11efc460de6 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1, ana group 2 context: None 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_513f8027-44a0-43dc-82c0-6df74017b797 from mypool/myimage13 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 19 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.154202] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_513f8027-44a0-43dc-82c0-6df74017b797 rbd disk to lun 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_513f8027-44a0-43dc-82c0-6df74017b797 to nqn.2016-06.io.spdk:cnode1 with ANA group id 2 using NSID 13 and UUID 513f8027-44a0-43dc-82c0-6df74017b797 2024-05-21T12:43:42.180 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:43:42.181 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 from mypool/myimage35 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.181 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 17 2024-05-21T12:43:42.181 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.175286] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 rbd disk to lun 2024-05-21T12:43:42.181 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_f8bee28e-a7ba-43f6-9e41-90f574236e57 to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 15 and UUID f8bee28e-a7ba-43f6-9e41-90f574236e57 2024-05-21T12:43:42.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:42 smithi072 ceph-mon[33912]: pgmap v951: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:42.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:43:42.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 from mypool/myimage17 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 18 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.196293] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 rbd disk to lun 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 17 and UUID 6f2d5759-26b1-4a43-a40f-9a4ef53b5a93 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2, ana group 1 context: None 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 from mypool/myimage36 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 19 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.217375] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 rbd disk to lun 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_b9f77c65-3e74-4b5e-9e44-0bbc82231e40 to nqn.2016-06.io.spdk:cnode2 with ANA group id 1 using NSID 16 and UUID b9f77c65-3e74-4b5e-9e44-0bbc82231e40 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1, ana group 3 context: None 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 from mypool/myimage11 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 19 2024-05-21T12:43:42.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.240281] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 rbd disk to lun 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_f4c0a98c-8cd4-427d-b870-36c81cc89c85 to nqn.2016-06.io.spdk:cnode1 with ANA group id 3 using NSID 11 and UUID f4c0a98c-8cd4-427d-b870-36c81cc89c85 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3, ana group 1 context: None 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 from mypool/myimage42 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_1_0' number bdevs: 20 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.260907] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 rbd disk to lun 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 to nqn.2016-06.io.spdk:cnode3 with ANA group id 1 using NSID 2 and UUID c2aaf5f6-e955-4853-b5d2-d4d9e8d86558 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3, ana group 2 context: None 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_37f162b2-e7c8-44a7-b545-84d228432f2b from mypool/myimage55 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_2_0' number bdevs: 20 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.282071] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b rbd disk to lun 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_37f162b2-e7c8-44a7-b545-84d228432f2b to nqn.2016-06.io.spdk:cnode3 with ANA group id 2 using NSID 15 and UUID 37f162b2-e7c8-44a7-b545-84d228432f2b 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:914: Received request to add a namespace using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2, ana group 3 context: None 2024-05-21T12:43:42.505 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:274: Received request to create bdev bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc from mypool/myimage23 (size 0 bytes) with block size 512, will not create image if doesn't exist 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:209: get_cluster cluster_name='cluster_context_3_0' number bdevs: 20 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.304239] bdev_rbd.c:1438:bdev_rbd_create: *NOTICE*: Add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc rbd disk to lun 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:760: Received request to add bdev_4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc to nqn.2016-06.io.spdk:cnode2 with ANA group id 3 using NSID 3 and UUID 4dbc9214-2d75-4dd3-850f-ae87ff8b7ddc 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode1, context: None 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode2, context: None 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:1755: Received request to allow any host access for nqn.2016-06.io.spdk:cnode3, context: None 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.92:4420, context: None 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:42.340760] tcp.c: 951:nvmf_tcp_listen: *NOTICE*: *** NVMe/TCP Target Listening on 172.21.15.92 port 4420 *** 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.45:4420, context: None 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.92:4420, context: None 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.45:4420, context: None 2024-05-21T12:43:42.506 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode3 at 172.21.15.72:4420, context: None 2024-05-21T12:43:42.507 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi045 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.45:4420, context: None 2024-05-21T12:43:42.507 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.72:4420, context: None 2024-05-21T12:43:42.507 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi092 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode1 at 172.21.15.92:4420, context: None 2024-05-21T12:43:42.507 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:42] INFO grpc.py:2099: Received request to create smithi072 TCP ipv4 listener for nqn.2016-06.io.spdk:cnode2 at 172.21.15.72:4420, context: None 2024-05-21T12:43:42.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:42 smithi045 ceph-mon[30165]: pgmap v951: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:43:43.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:43:43] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:43:43.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:43:43.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:43:43.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:43:43.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:43:43.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:43:43.567 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:43:43.568 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: ], , client address: IPv4 172.21.15.45:40012 2024-05-21T12:43:44.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:44 smithi072 ceph-mon[33912]: pgmap v952: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 2.5 KiB/s wr, 80 op/s 2024-05-21T12:43:44.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:44 smithi092 ceph-mon[33665]: pgmap v952: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 2.5 KiB/s wr, 80 op/s 2024-05-21T12:43:44.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:44 smithi045 ceph-mon[30165]: pgmap v952: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 175 KiB/s rd, 2.5 KiB/s wr, 80 op/s 2024-05-21T12:43:46.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:46 smithi072 ceph-mon[33912]: pgmap v953: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 268 KiB/s rd, 4.1 KiB/s wr, 136 op/s 2024-05-21T12:43:46.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-mon[33665]: pgmap v953: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 268 KiB/s rd, 4.1 KiB/s wr, 136 op/s 2024-05-21T12:43:46.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:43:46] INFO grpc.py:822: Received request to set ana states [nqn: "nqn.2016-06.io.spdk:cnode1" 2024-05-21T12:43:46.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:46.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 3 2024-05-21T12:43:46.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: OPTIMIZED 2024-05-21T12:43:46.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:46.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: , nqn: "nqn.2016-06.io.spdk:cnode2" 2024-05-21T12:43:46.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:46.503 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 3 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: OPTIMIZED 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 3 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: OPTIMIZED 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: ], , client address: IPv4 172.21.15.92:45706 2024-05-21T12:43:46.504 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:46.153018] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000266 seconds 2024-05-21T12:43:46.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:46 smithi045 ceph-mon[30165]: pgmap v953: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 268 KiB/s rd, 4.1 KiB/s wr, 136 op/s 2024-05-21T12:43:48.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:48 smithi072 ceph-mon[33912]: pgmap v954: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 338 KiB/s rd, 5.0 KiB/s wr, 167 op/s 2024-05-21T12:43:48.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:48 smithi092 ceph-mon[33665]: pgmap v954: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 338 KiB/s rd, 5.0 KiB/s wr, 167 op/s 2024-05-21T12:43:48.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:48 smithi045 ceph-mon[30165]: pgmap v954: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 338 KiB/s rd, 5.0 KiB/s wr, 167 op/s 2024-05-21T12:43:50.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:50 smithi072 ceph-mon[33912]: pgmap v955: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.7 KiB/s wr, 208 op/s 2024-05-21T12:43:50.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:50 smithi092 ceph-mon[33665]: pgmap v955: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.7 KiB/s wr, 208 op/s 2024-05-21T12:43:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:50 smithi045 ceph-mon[30165]: pgmap v955: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.3 MiB/s rd, 5.7 KiB/s wr, 208 op/s 2024-05-21T12:43:52.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:52 smithi072 ceph-mon[33912]: pgmap v956: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 6.8 KiB/s wr, 238 op/s 2024-05-21T12:43:52.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:52 smithi092 ceph-mon[33665]: pgmap v956: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 6.8 KiB/s wr, 238 op/s 2024-05-21T12:43:52.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:52 smithi045 ceph-mon[30165]: pgmap v956: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.0 MiB/s rd, 6.8 KiB/s wr, 238 op/s 2024-05-21T12:43:54.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:54 smithi072 ceph-mon[33912]: pgmap v957: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 8.3 KiB/s wr, 279 op/s 2024-05-21T12:43:54.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:54 smithi092 ceph-mon[33665]: pgmap v957: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 8.3 KiB/s wr, 279 op/s 2024-05-21T12:43:54.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:54 smithi045 ceph-mon[30165]: pgmap v957: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.9 MiB/s rd, 8.3 KiB/s wr, 279 op/s 2024-05-21T12:43:56.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:56 smithi072 ceph-mon[33912]: pgmap v958: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 5.8 KiB/s wr, 200 op/s 2024-05-21T12:43:56.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:56 smithi092 ceph-mon[33665]: pgmap v958: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 5.8 KiB/s wr, 200 op/s 2024-05-21T12:43:56.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:56 smithi045 ceph-mon[30165]: pgmap v958: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.7 MiB/s rd, 5.8 KiB/s wr, 200 op/s 2024-05-21T12:43:58.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:43:58 smithi072 ceph-mon[33912]: pgmap v959: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 4.2 KiB/s wr, 144 op/s 2024-05-21T12:43:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:43:58 smithi092 ceph-mon[33665]: pgmap v959: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 4.2 KiB/s wr, 144 op/s 2024-05-21T12:43:58.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:43:58 smithi045 ceph-mon[30165]: pgmap v959: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 4.2 KiB/s wr, 144 op/s 2024-05-21T12:44:00.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:00 smithi072 ceph-mon[33912]: pgmap v960: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.3 KiB/s wr, 113 op/s 2024-05-21T12:44:00.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:00 smithi092 ceph-mon[33665]: pgmap v960: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.3 KiB/s wr, 113 op/s 2024-05-21T12:44:00.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:00 smithi045 ceph-mon[30165]: pgmap v960: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.3 KiB/s wr, 113 op/s 2024-05-21T12:44:02.416 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:02 smithi072 ceph-mon[33912]: pgmap v961: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.6 KiB/s wr, 72 op/s 2024-05-21T12:44:02.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:02 smithi092 ceph-mon[33665]: pgmap v961: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.6 KiB/s wr, 72 op/s 2024-05-21T12:44:02.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:02 smithi045 ceph-mon[30165]: pgmap v961: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.6 KiB/s wr, 72 op/s 2024-05-21T12:44:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:04 smithi092 ceph-mon[33665]: pgmap v962: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 952 KiB/s rd, 1.5 KiB/s wr, 42 op/s 2024-05-21T12:44:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:04 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:44:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:04 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:44:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:04 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:44:04.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:04 smithi092 ceph-mon[33665]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:44:04.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:04 smithi045 ceph-mon[30165]: pgmap v962: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 952 KiB/s rd, 1.5 KiB/s wr, 42 op/s 2024-05-21T12:44:04.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:04 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:44:04.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:04 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:44:04.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:04 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:44:04.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:04 smithi045 ceph-mon[30165]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:44:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:04 smithi072 ceph-mon[33912]: pgmap v962: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 952 KiB/s rd, 1.5 KiB/s wr, 42 op/s 2024-05-21T12:44:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:04 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-05-21T12:44:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:04 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-05-21T12:44:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:04 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-05-21T12:44:04.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:04 smithi072 ceph-mon[33912]: from='mgr.14152 172.21.15.45:0/3527540790' entity='mgr.x' 2024-05-21T12:44:05.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:05 smithi092 ceph-mon[33665]: pgmap v963: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 25 KiB/s rd, 2 op/s 2024-05-21T12:44:05.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:05 smithi092 ceph-mon[33665]: pgmap v964: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:05.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:05 smithi045 ceph-mon[30165]: pgmap v963: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 25 KiB/s rd, 2 op/s 2024-05-21T12:44:05.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:05 smithi045 ceph-mon[30165]: pgmap v964: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:05.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:05 smithi072 ceph-mon[33912]: pgmap v963: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 25 KiB/s rd, 2 op/s 2024-05-21T12:44:05.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:05 smithi072 ceph-mon[33912]: pgmap v964: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:07.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:07 smithi092 ceph-mon[33665]: pgmap v965: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:44:07.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:07 smithi045 ceph-mon[30165]: pgmap v965: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:44:07.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:07 smithi072 ceph-mon[33912]: pgmap v965: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 15 KiB/s rd, 1 op/s 2024-05-21T12:44:09.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:09 smithi092 ceph-mon[33665]: pgmap v966: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:09.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:09 smithi045 ceph-mon[30165]: pgmap v966: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:09.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:09 smithi072 ceph-mon[33912]: pgmap v966: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:11.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:11 smithi092 ceph-mon[33665]: pgmap v967: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:11.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:11 smithi045 ceph-mon[30165]: pgmap v967: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:11.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:11 smithi072 ceph-mon[33912]: pgmap v967: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:13.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:13 smithi092 ceph-mon[33665]: pgmap v968: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:13.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:13 smithi045 ceph-mon[30165]: pgmap v968: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:13.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:13 smithi072 ceph-mon[33912]: pgmap v968: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 31 KiB/s rd, 2 op/s 2024-05-21T12:44:15.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:15 smithi092 ceph-mon[33665]: pgmap v969: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 KiB/s rd, 2 op/s 2024-05-21T12:44:15.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:15 smithi045 ceph-mon[30165]: pgmap v969: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 KiB/s rd, 2 op/s 2024-05-21T12:44:15.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:15 smithi072 ceph-mon[33912]: pgmap v969: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 28 KiB/s rd, 2 op/s 2024-05-21T12:44:17.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:17 smithi092 ceph-mon[33665]: pgmap v970: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:17.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:17 smithi045 ceph-mon[30165]: pgmap v970: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:17.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:17 smithi072 ceph-mon[33912]: pgmap v970: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:19.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:19 smithi092 ceph-mon[33665]: pgmap v971: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:19.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:19 smithi045 ceph-mon[30165]: pgmap v971: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:19.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:19 smithi072 ceph-mon[33912]: pgmap v971: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:21.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:21 smithi092 ceph-mon[33665]: pgmap v972: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:21.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:21 smithi045 ceph-mon[30165]: pgmap v972: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:21.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:21 smithi072 ceph-mon[33912]: pgmap v972: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:23.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:23 smithi092 ceph-mon[33665]: pgmap v973: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:23.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:23 smithi045 ceph-mon[30165]: pgmap v973: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:23.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:23 smithi072 ceph-mon[33912]: pgmap v973: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:25.468 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:25 smithi072 ceph-mon[33912]: pgmap v974: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:25.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:25 smithi092 ceph-mon[33665]: pgmap v974: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:25.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:25 smithi045 ceph-mon[30165]: pgmap v974: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:27.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:27 smithi092 ceph-mon[33665]: pgmap v975: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:27.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:27 smithi045 ceph-mon[30165]: pgmap v975: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:27.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:27 smithi072 ceph-mon[33912]: pgmap v975: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:29.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:29 smithi092 ceph-mon[33665]: pgmap v976: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:29.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:29 smithi045 ceph-mon[30165]: pgmap v976: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:29.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:29 smithi072 ceph-mon[33912]: pgmap v976: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:31.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:31 smithi092 ceph-mon[33665]: pgmap v977: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:31.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:31 smithi045 ceph-mon[30165]: pgmap v977: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:31.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:31 smithi072 ceph-mon[33912]: pgmap v977: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:33.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:33 smithi092 ceph-mon[33665]: pgmap v978: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:33.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:33 smithi045 ceph-mon[30165]: pgmap v978: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:33.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:33 smithi072 ceph-mon[33912]: pgmap v978: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:35.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:35 smithi092 ceph-mon[33665]: pgmap v979: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:35.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:35 smithi045 ceph-mon[30165]: pgmap v979: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:35.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:35 smithi072 ceph-mon[33912]: pgmap v979: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:37.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:37 smithi092 ceph-mon[33665]: pgmap v980: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:37.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:37 smithi045 ceph-mon[30165]: pgmap v980: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:37.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:37 smithi072 ceph-mon[33912]: pgmap v980: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:39.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:39 smithi092 ceph-mon[33665]: pgmap v981: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:39.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:39 smithi045 ceph-mon[30165]: pgmap v981: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:39.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:39 smithi072 ceph-mon[33912]: pgmap v981: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:41.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:41 smithi092 ceph-mon[33665]: pgmap v982: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:41.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:41 smithi045 ceph-mon[30165]: pgmap v982: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:41.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:41 smithi072 ceph-mon[33912]: pgmap v982: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:43.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:43 smithi092 ceph-mon[33665]: pgmap v983: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:43.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:43 smithi045 ceph-mon[30165]: pgmap v983: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:43.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:43 smithi072 ceph-mon[33912]: pgmap v983: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:45.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:45 smithi092 ceph-mon[33665]: pgmap v984: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:45.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:45 smithi045 ceph-mon[30165]: pgmap v984: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:45.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:45 smithi072 ceph-mon[33912]: pgmap v984: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:47.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:47 smithi092 ceph-mon[33665]: pgmap v985: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:47.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:47 smithi045 ceph-mon[30165]: pgmap v985: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:47.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:47 smithi072 ceph-mon[33912]: pgmap v985: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:49.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:49 smithi092 ceph-mon[33665]: pgmap v986: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:49.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:49 smithi045 ceph-mon[30165]: pgmap v986: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:49.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:49 smithi072 ceph-mon[33912]: pgmap v986: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:51.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:51 smithi092 ceph-mon[33665]: pgmap v987: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:51.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:51 smithi045 ceph-mon[30165]: pgmap v987: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:51.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:51 smithi072 ceph-mon[33912]: pgmap v987: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:53.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:53 smithi092 ceph-mon[33665]: pgmap v988: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:53.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:53 smithi045 ceph-mon[30165]: pgmap v988: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:53.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:53 smithi072 ceph-mon[33912]: pgmap v988: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:55.468 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:55 smithi072 ceph-mon[33912]: pgmap v989: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:55.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:55 smithi092 ceph-mon[33665]: pgmap v989: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:55.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:55 smithi045 ceph-mon[30165]: pgmap v989: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:56.144 INFO:tasks.nvmeof.[nvmeof.thrasher]:done waiting before thrashing 2024-05-21T12:44:56.144 INFO:tasks.nvmeof.[nvmeof.thrasher]:display and verify stats: 2024-05-21T12:44:56.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a 2024-05-21T12:44:56.180 INFO:teuthology.orchestra.run.smithi045.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service - Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: Active: active (running) since Tue 2024-05-21 12:39:51 UTC; 5min ago 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76953 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: Process: 76954 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.a/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: Main PID: 77017 (conmon) 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: Tasks: 126 (limit: 202854) 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: Memory: 156.4M 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: CPU: 5min 7.601s 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service 2024-05-21T12:44:56.181 INFO:teuthology.orchestra.run.smithi045.stdout: ├─libpod-payload-40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77019 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77021 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77039 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.a --gateway-address 172.21.15.45:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.45:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.a -k /etc/ceph/keyring 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout: │ ├─77090 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout: │ └─77886 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout: └─runtime 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout: └─77017 /usr/bin/conmon --api-version 1 -c 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f -u 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata -p /run/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:44:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:44:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:44:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:44:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: states { 2024-05-21T12:44:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: grp_id: 3 2024-05-21T12:44:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: state: INACCESSIBLE 2024-05-21T12:44:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: } 2024-05-21T12:44:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:May 21 12:43:43 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: ], , client address: IPv4 172.21.15.45:40012 2024-05-21T12:44:56.184 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b 2024-05-21T12:44:56.214 INFO:teuthology.orchestra.run.smithi072.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service - Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:44:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:44:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: Active: active (running) since Tue 2024-05-21 12:36:12 UTC; 8min ago 2024-05-21T12:44:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56908 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:44:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: Process: 56909 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.b/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:44:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: Main PID: 56971 (conmon) 2024-05-21T12:44:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: Tasks: 126 (limit: 202814) 2024-05-21T12:44:56.214 INFO:teuthology.orchestra.run.smithi072.stdout: Memory: 172.0M 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: CPU: 8min 53.847s 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: ├─libpod-payload-8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56973 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56975 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─56993 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.b --gateway-address 172.21.15.72:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.72:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.b -k /etc/ceph/keyring 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: │ ├─57044 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: │ └─57834 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: └─runtime 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: └─56971 /usr/bin/conmon --api-version 1 -c 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -u 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata -p /run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-21T12:44:56.215 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: states { 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: grp_id: 3 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: state: INACCESSIBLE 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: } 2024-05-21T12:44:56.216 INFO:teuthology.orchestra.run.smithi072.stdout:May 21 12:39:54 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: ], , client address: IPv4 172.21.15.72:51082 2024-05-21T12:44:56.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl status ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c 2024-05-21T12:44:56.250 INFO:teuthology.orchestra.run.smithi092.stdout:● ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service - Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:44:56.250 INFO:teuthology.orchestra.run.smithi092.stdout: Loaded: loaded (/etc/systemd/system/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@.service; enabled; preset: disabled) 2024-05-21T12:44:56.250 INFO:teuthology.orchestra.run.smithi092.stdout: Active: active (running) since Tue 2024-05-21 12:43:40 UTC; 1min 16s ago 2024-05-21T12:44:56.250 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 60171 ExecStartPre=/bin/rm -f /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-cid (code=exited, status=0/SUCCESS) 2024-05-21T12:44:56.250 INFO:teuthology.orchestra.run.smithi092.stdout: Process: 60172 ExecStart=/bin/bash /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/nvmeof.nvmeof.c/unit.run (code=exited, status=0/SUCCESS) 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: Main PID: 60232 (conmon) 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: Tasks: 126 (limit: 202816) 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: Memory: 142.8M 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: CPU: 1min 16.518s 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: CGroup: /system.slice/system-ceph\x2dc7e3752a\x2d176b\x2d11ef\x2dbc9a\x2dc7b262605968.slice/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: ├─libpod-payload-062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─60234 /run/podman-init -- python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─60236 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─60254 /usr/bin/ceph-nvmeof-monitor-client --gateway-name client.nvmeof.nvmeof.c --gateway-address 172.21.15.92:5500 --gateway-pool mypool --gateway-group "" --monitor-group-address 172.21.15.92:5499 -c /etc/ceph/ceph.conf -n client.nvmeof.nvmeof.c -k /etc/ceph/keyring 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: │ ├─60305 /usr/local/bin/nvmf_tgt -u -r /var/tmp/spdk.sock 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: │ └─61128 python3 -m control -c /src/ceph-nvmeof.conf 2024-05-21T12:44:56.251 INFO:teuthology.orchestra.run.smithi092.stdout: └─runtime 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout: └─60232 /usr/bin/conmon --api-version 1 -c 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 -u 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4/userdata -p /run/containers/storage/overlay-containers/062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4/userdata/pidfile -n ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c --exit-dir /run/libpod/exits --persist-dir /run/libpod/persist/062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 --full-attach -l journald --log-level warning --syslog --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4/userdata/oci-log --conmon-pidfile /run/ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service-pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 3 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: OPTIMIZED 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: , nqn: "nqn.2016-06.io.spdk:cnode3" 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: states { 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: grp_id: 3 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: state: OPTIMIZED 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: } 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: ], , client address: IPv4 172.21.15.92:45706 2024-05-21T12:44:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:May 21 12:43:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [2024-05-21 12:43:46.153018] bdev_rbd.c:1039:_bdev_rbd_wait_for_latest_osdmap: *ERROR*: rados_wait_for_latest_osdmap cluster: cluster_context_3_0 elapsed time: 0.000266 seconds 2024-05-21T12:44:56.254 DEBUG:teuthology.orchestra.run.smithi137:> ceph orch ls && ceph orch ps --daemon-type nvmeof && ceph health detail && ceph -s && sudo nvme list-subsys /dev/nvme5n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n1 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n2 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n3 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n4 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n5 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n6 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n7 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n8 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n9 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n10 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n11 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n12 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n13 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n14 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n15 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n16 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n17 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n18 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n19 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme5n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme2n20 | grep 'live optimized' && sudo nvme list-subsys /dev/nvme8n20 | grep 'live optimized' 2024-05-21T12:44:56.560 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T12:44:56.560 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 1/1 10m ago 26m smithi045=x;count:1 2024-05-21T12:44:56.560 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 10m ago 26m smithi045:172.21.15.45=a;smithi072:172.21.15.72=b;smithi092:172.21.15.92=c;count:3 2024-05-21T12:44:56.560 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.mypool ?:4420,5500,8009 1/3 10m ago 21m smithi045=nvmeof.a;smithi072=nvmeof.b;smithi092=nvmeof.c;count:3 2024-05-21T12:44:56.560 INFO:teuthology.orchestra.run.smithi137.stdout:osd 8 10m ago - 2024-05-21T12:44:56.851 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T12:44:56.852 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.a smithi045 *:5500,4420,8009 error 10m ago 20m - - 2024-05-21T12:44:56.852 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.b smithi072 *:5500,4420,8009 error 10m ago 20m - - 2024-05-21T12:44:56.852 INFO:teuthology.orchestra.run.smithi137.stdout:nvmeof.nvmeof.c smithi092 *:5500,4420,8009 running (21m) 10m ago 20m 160M - 1.2.9 30dd525ace30 5e5a8ffd2ba6 2024-05-21T12:44:57.199 INFO:teuthology.orchestra.run.smithi137.stdout:HEALTH_WARN 2 failed cephadm daemon(s) 2024-05-21T12:44:57.199 INFO:teuthology.orchestra.run.smithi137.stdout:[WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-05-21T12:44:57.199 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.a on smithi045 is in error state 2024-05-21T12:44:57.199 INFO:teuthology.orchestra.run.smithi137.stdout: daemon nvmeof.nvmeof.b on smithi072 is in error state 2024-05-21T12:44:57.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:57 smithi045 ceph-mon[30165]: pgmap v990: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:57.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:57 smithi045 ceph-mon[30165]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:44:57.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:57 smithi045 ceph-mon[30165]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:44:57.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:57 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/2919687240' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:44:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:57 smithi092 ceph-mon[33665]: pgmap v990: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:57 smithi092 ceph-mon[33665]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:44:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:57 smithi092 ceph-mon[33665]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:44:57.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:57 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/2919687240' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: cluster: 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: id: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: health: HEALTH_WARN 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: 2 failed cephadm daemon(s) 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: services: 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: mon: 3 daemons, quorum a,c,b (age 26m) 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: mgr: x(active, since 27m) 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: osd: 8 osds: 8 up (since 22m), 8 in (since 23m) 2024-05-21T12:44:57.552 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: data: 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: pools: 2 pools, 33 pgs 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: objects: 485 objects, 1.0 GiB 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: usage: 4.3 GiB used, 711 GiB / 715 GiB avail 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: pgs: 33 active+clean 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: io: 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: client: 23 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-05-21T12:44:57.553 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-21T12:44:57.582 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.601 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.619 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.638 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.656 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:57 smithi072 ceph-mon[33912]: pgmap v990: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 23 KiB/s rd, 1 op/s 2024-05-21T12:44:57.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:57 smithi072 ceph-mon[33912]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:44:57.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:57 smithi072 ceph-mon[33912]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "daemon_type": "nvmeof", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T12:44:57.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:57 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/2919687240' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-05-21T12:44:57.674 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.693 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.712 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.730 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.750 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.768 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.786 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.804 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.823 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.841 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.933 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.952 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.970 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:57.989 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.009 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.026 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.044 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.064 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.082 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.100 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.118 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.136 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.155 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.173 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.192 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.211 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.229 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.248 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.266 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.285 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.303 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.321 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.340 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.358 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.377 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.395 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.413 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.431 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.450 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.468 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.487 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:58 smithi092 ceph-mon[33665]: from='client.? 172.21.15.137:0/2275346244' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:44:58.505 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.524 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme7 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.542 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme2 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.560 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme9 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:58 smithi045 ceph-mon[30165]: from='client.? 172.21.15.137:0/2275346244' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:44:58.578 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme5 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.597 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme3 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.616 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme10 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.634 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme6 tcp traddr=172.21.15.72,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.652 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme4 tcp traddr=172.21.15.92,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:58 smithi072 ceph-mon[33912]: from='client.? 172.21.15.137:0/2275346244' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-05-21T12:44:58.671 INFO:teuthology.orchestra.run.smithi137.stdout: +- nvme8 tcp traddr=172.21.15.45,trsvcid=4420,src_addr=172.21.15.137 live optimized 2024-05-21T12:44:58.671 INFO:tasks.nvmeof.[nvmeof.thrasher]:Thrasher summary: 2024-05-21T12:44:58.672 INFO:tasks.nvmeof.[nvmeof.thrasher]:nvmeof.a was thrashed 3 times 2024-05-21T12:44:58.672 INFO:tasks.nvmeof.[nvmeof.thrasher]:nvmeof.b was thrashed 1 times 2024-05-21T12:44:58.672 INFO:tasks.nvmeof.[nvmeof.thrasher]:nvmeof.c was thrashed 2 times 2024-05-21T12:44:58.672 INFO:tasks.nvmeof.[nvmeof.thrasher]:Iteration 0: killed: nvmeof.a 2024-05-21T12:44:58.672 INFO:tasks.nvmeof.[nvmeof.thrasher]:Iteration 1: killed: nvmeof.a, nvmeof.b 2024-05-21T12:44:58.672 INFO:tasks.nvmeof.[nvmeof.thrasher]:Iteration 2: killed: nvmeof.a, nvmeof.c 2024-05-21T12:44:58.672 INFO:tasks.nvmeof.[nvmeof.thrasher]:Iteration 3: killed: nvmeof.c 2024-05-21T12:44:58.672 INFO:tasks.nvmeof:done joining 2024-05-21T12:44:58.672 DEBUG:teuthology.run_tasks:Unwinding manager nvmeof 2024-05-21T12:44:58.682 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-21T12:44:58.691 INFO:tasks.cephadm:Teardown begin 2024-05-21T12:44:58.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:44:58.718 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:44:58.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:44:58.770 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:44:58.794 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-21T12:44:58.795 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T12:44:58.810 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T12:44:58.826 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T12:44:58.841 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T12:44:58.856 INFO:tasks.cephadm:Stopping all daemons... 2024-05-21T12:44:58.856 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-21T12:44:58.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.a 2024-05-21T12:44:59.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:58 smithi045 systemd[1]: Stopping Ceph mon.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:44:59.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:44:59 smithi092 ceph-mon[33665]: pgmap v991: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:59 smithi045 ceph-mon[30165]: pgmap v991: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:44:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a[30161]: 2024-05-21T12:44:59.302+0000 7fa1b3f29700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:44:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:59 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a[30161]: 2024-05-21T12:44:59.302+0000 7fa1b3f29700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-21T12:44:59.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:59 smithi045 podman[79313]: 2024-05-21 12:44:59.394571397 +0000 UTC m=+0.326671619 container died 80d88adb7cd3fb8ffc9a79f419c020917186deff3a8ad2862cff51735a3090b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=ceph-nvmeof-mon-b0c764b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-05-21T12:44:59.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:44:59 smithi072 ceph-mon[33912]: pgmap v991: 33 pgs: 33 active+clean; 1.0 GiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 34 KiB/s rd, 2 op/s 2024-05-21T12:45:00.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:59 smithi045 podman[79313]: 2024-05-21 12:44:59.870539368 +0000 UTC m=+0.802639589 container cleanup 80d88adb7cd3fb8ffc9a79f419c020917186deff3a8ad2862cff51735a3090b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b, CEPH_POINT_RELEASE=) 2024-05-21T12:45:00.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:May 21 12:44:59 smithi045 bash[79313]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-a 2024-05-21T12:45:00.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-21T12:45:00.298 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:00.298 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-21T12:45:00.299 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-21T12:45:00.299 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.b 2024-05-21T12:45:00.389 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:45:00 smithi072 systemd[1]: Stopping Ceph mon.b for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:45:00 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-b[33908]: 2024-05-21T12:45:00.862+0000 7fc498629700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:45:00 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-b[33908]: 2024-05-21T12:45:00.862+0000 7fc498629700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-21T12:45:01.166 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:45:00 smithi072 podman[60346]: 2024-05-21 12:45:00.944488874 +0000 UTC m=+0.185089588 container died a8904b85645486aea02151a6808625532dec6b9bc8692ad12a8bbf9e6fe34c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-21T12:45:01.662 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:45:01 smithi072 podman[60346]: 2024-05-21 12:45:01.415936717 +0000 UTC m=+0.656537422 container cleanup a8904b85645486aea02151a6808625532dec6b9bc8692ad12a8bbf9e6fe34c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-21T12:45:01.663 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 21 12:45:01 smithi072 bash[60346]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-b 2024-05-21T12:45:01.847 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-21T12:45:01.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:01.885 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-21T12:45:01.885 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-21T12:45:01.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.c 2024-05-21T12:45:02.187 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:01 smithi092 systemd[1]: Stopping Ceph mon.c for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:02 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c[33648]: 2024-05-21T12:45:02.185+0000 7f52c4b93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:02 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c[33648]: 2024-05-21T12:45:02.185+0000 7f52c4b93700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-21T12:45:02.503 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:02 smithi092 podman[61902]: 2024-05-21 12:45:02.348789761 +0000 UTC m=+0.275801608 container died 9de20c30070169f0e6746e116b2a2bde09001ea26f6dd900bc3bf8fd84305376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T12:45:02.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-05-21T12:45:02.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-05-21T12:45:02.971 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:02 smithi092 podman[61902]: 2024-05-21 12:45:02.769667101 +0000 UTC m=+0.696678951 container cleanup 9de20c30070169f0e6746e116b2a2bde09001ea26f6dd900bc3bf8fd84305376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T12:45:02.972 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:02 smithi092 bash[61902]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c 2024-05-21T12:45:03.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:02 smithi092 podman[61914]: 2024-05-21 12:45:02.971419147 +0000 UTC m=+0.618575099 container remove 9de20c30070169f0e6746e116b2a2bde09001ea26f6dd900bc3bf8fd84305376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-21T12:45:03.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:03 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.c.service: Deactivated successfully. 2024-05-21T12:45:03.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:03 smithi092 systemd[1]: Stopped Ceph mon.c for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:45:03.222 INFO:journalctl@ceph.mon.c.smithi092.stdout:May 21 12:45:03 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.c.service: Consumed 9.689s CPU time. 2024-05-21T12:45:03.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-21T12:45:03.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:03.261 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-21T12:45:03.261 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-21T12:45:03.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T12:45:03.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:45:03 smithi045 systemd[1]: Stopping Ceph mgr.x for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:03.915 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:45:03 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x[30395]: 2024-05-21T12:45:03.605+0000 7f83f2eff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:03.916 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:45:03 smithi045 podman[79719]: 2024-05-21 12:45:03.63714775 +0000 UTC m=+0.159778176 container died 87ed08596cafcf6caf1e89600d9ab3c6bf276dbeed48c8f449496c8163af149d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux ) 2024-05-21T12:45:04.307 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:45:03 smithi045 podman[79719]: 2024-05-21 12:45:03.989051308 +0000 UTC m=+0.511681736 container cleanup 87ed08596cafcf6caf1e89600d9ab3c6bf276dbeed48c8f449496c8163af149d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b0c764b6e22fbb51f59e469d9dd99895c152f73e, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-21T12:45:04.308 INFO:journalctl@ceph.mgr.x.smithi045.stdout:May 21 12:45:03 smithi045 bash[79719]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-mgr-x 2024-05-21T12:45:04.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-21T12:45:04.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:04.508 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-21T12:45:04.508 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-21T12:45:04.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.0 2024-05-21T12:45:04.836 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:04 smithi045 systemd[1]: Stopping Ceph osd.0 for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:05.317 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0[44772]: 2024-05-21T12:45:04.834+0000 7f0a5995e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:05.317 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0[44772]: 2024-05-21T12:45:04.834+0000 7f0a5995e700 -1 osd.0 163 *** Got signal Terminated *** 2024-05-21T12:45:05.317 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:04 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0[44772]: 2024-05-21T12:45:04.834+0000 7f0a5995e700 -1 osd.0 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T12:45:07.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~5s 2024-05-21T12:45:07.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~5s 2024-05-21T12:45:07.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-05-21T12:45:07.787 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-05-21T12:45:10.528 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:10 smithi045 podman[79842]: 2024-05-21 12:45:10.232811458 +0000 UTC m=+5.489851448 container died c40f577782422fe084e509620f414c5f35b83e7ab1424aa9290c1534bbe54f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-21T12:45:10.812 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:10 smithi045 podman[79842]: 2024-05-21 12:45:10.627107436 +0000 UTC m=+5.884147415 container cleanup c40f577782422fe084e509620f414c5f35b83e7ab1424aa9290c1534bbe54f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-21T12:45:10.812 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:10 smithi045 bash[79842]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0 2024-05-21T12:45:11.067 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:10 smithi045 podman[79867]: 2024-05-21 12:45:10.811860617 +0000 UTC m=+0.578335417 container remove c40f577782422fe084e509620f414c5f35b83e7ab1424aa9290c1534bbe54f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=ceph-nvmeof-mon-b0c764b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T12:45:11.616 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:11 smithi045 podman[79956]: 2024-05-21 12:45:11.232967116 +0000 UTC m=+0.018503169 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:45:11.616 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:11 smithi045 podman[79956]: 2024-05-21 12:45:11.3377004 +0000 UTC m=+0.123236447 container create 890f2778996133d91be140c6154f1330592a86001d3e85f46c87700a45894a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=ceph-nvmeof-mon-b0c764b, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T12:45:11.616 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:11 smithi045 podman[79956]: 2024-05-21 12:45:11.557196339 +0000 UTC m=+0.342732401 container init 890f2778996133d91be140c6154f1330592a86001d3e85f46c87700a45894a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0-deactivate, RELEASE=ceph-nvmeof-mon-b0c764b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-21T12:45:11.616 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:11 smithi045 podman[79956]: 2024-05-21 12:45:11.560790538 +0000 UTC m=+0.346326584 container start 890f2778996133d91be140c6154f1330592a86001d3e85f46c87700a45894a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=ceph-nvmeof-mon-b0c764b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-21T12:45:11.616 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:11 smithi045 podman[79956]: 2024-05-21 12:45:11.615145789 +0000 UTC m=+0.400681853 container attach 890f2778996133d91be140c6154f1330592a86001d3e85f46c87700a45894a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-21T12:45:12.168 INFO:journalctl@ceph.osd.0.smithi045.stdout:May 21 12:45:11 smithi045 podman[79956]: 2024-05-21 12:45:11.870179323 +0000 UTC m=+0.655715367 container died 890f2778996133d91be140c6154f1330592a86001d3e85f46c87700a45894a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T12:45:12.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-21T12:45:12.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:12.490 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-21T12:45:12.490 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-21T12:45:12.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.1 2024-05-21T12:45:12.729 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:12 smithi045 systemd[1]: Stopping Ceph osd.1 for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:12.787 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-21T12:45:12.788 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~10s 2024-05-21T12:45:12.788 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~10s 2024-05-21T12:45:12.788 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~5s 2024-05-21T12:45:12.788 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~5s 2024-05-21T12:45:13.067 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1[50681]: 2024-05-21T12:45:12.829+0000 7fd0ef5be700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:13.067 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1[50681]: 2024-05-21T12:45:12.829+0000 7fd0ef5be700 -1 osd.1 163 *** Got signal Terminated *** 2024-05-21T12:45:13.067 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:12 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1[50681]: 2024-05-21T12:45:12.829+0000 7fd0ef5be700 -1 osd.1 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T12:45:17.789 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2024-05-21T12:45:17.789 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~15s 2024-05-21T12:45:17.790 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~15s 2024-05-21T12:45:17.790 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~10s 2024-05-21T12:45:17.790 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~10s 2024-05-21T12:45:18.447 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:18 smithi045 podman[80062]: 2024-05-21 12:45:18.196497677 +0000 UTC m=+5.466012567 container died fcded796e72bba102884cd463437a421cca422c01896699d90adbd12d6c2bf82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-21T12:45:18.817 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:18 smithi045 podman[80062]: 2024-05-21 12:45:18.538129059 +0000 UTC m=+5.807643942 container cleanup fcded796e72bba102884cd463437a421cca422c01896699d90adbd12d6c2bf82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-21T12:45:18.817 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:18 smithi045 bash[80062]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1 2024-05-21T12:45:18.817 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:18 smithi045 podman[80100]: 2024-05-21 12:45:18.680945775 +0000 UTC m=+0.483593415 container remove fcded796e72bba102884cd463437a421cca422c01896699d90adbd12d6c2bf82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=ceph-nvmeof-mon-b0c764b) 2024-05-21T12:45:19.476 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:19 smithi045 podman[80186]: 2024-05-21 12:45:19.074967941 +0000 UTC m=+0.020935573 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:45:19.477 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:19 smithi045 podman[80186]: 2024-05-21 12:45:19.190245443 +0000 UTC m=+0.136213069 container create b8a4fee8828e13d7fe3be769fbc18b78ccc8e0f8c47cc2257da8feff879464fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-21T12:45:19.477 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:19 smithi045 podman[80186]: 2024-05-21 12:45:19.384550973 +0000 UTC m=+0.330518606 container init b8a4fee8828e13d7fe3be769fbc18b78ccc8e0f8c47cc2257da8feff879464fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-21T12:45:19.477 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:19 smithi045 podman[80186]: 2024-05-21 12:45:19.387885535 +0000 UTC m=+0.333853160 container start b8a4fee8828e13d7fe3be769fbc18b78ccc8e0f8c47cc2257da8feff879464fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=ceph-nvmeof-mon-b0c764b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-21T12:45:19.477 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:19 smithi045 podman[80186]: 2024-05-21 12:45:19.476215236 +0000 UTC m=+0.422182861 container attach b8a4fee8828e13d7fe3be769fbc18b78ccc8e0f8c47cc2257da8feff879464fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=ceph-nvmeof-mon-b0c764b, GIT_CLEAN=True) 2024-05-21T12:45:19.817 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 21 12:45:19 smithi045 podman[80186]: 2024-05-21 12:45:19.691002076 +0000 UTC m=+0.636969701 container died b8a4fee8828e13d7fe3be769fbc18b78ccc8e0f8c47cc2257da8feff879464fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True) 2024-05-21T12:45:20.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-21T12:45:20.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:20.364 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-21T12:45:20.364 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-21T12:45:20.365 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.2 2024-05-21T12:45:20.666 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:20 smithi072 systemd[1]: Stopping Ceph osd.2 for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:21.166 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:20 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2[37514]: 2024-05-21T12:45:20.664+0000 7fcd0b632700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:21.166 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:20 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2[37514]: 2024-05-21T12:45:20.664+0000 7fcd0b632700 -1 osd.2 163 *** Got signal Terminated *** 2024-05-21T12:45:21.166 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:20 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2[37514]: 2024-05-21T12:45:20.664+0000 7fcd0b632700 -1 osd.2 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T12:45:22.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~10s 2024-05-21T12:45:22.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-21T12:45:22.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~20s 2024-05-21T12:45:22.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~20s 2024-05-21T12:45:22.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~15s 2024-05-21T12:45:22.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~15s 2024-05-21T12:45:26.362 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:26 smithi072 podman[60531]: 2024-05-21 12:45:26.081046884 +0000 UTC m=+5.521741411 container died ffe9aa9c8813136bbd113eb0581bea8185c52e8fc471ccbcc56927ac37f59c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-21T12:45:26.666 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:26 smithi072 podman[60531]: 2024-05-21 12:45:26.469746706 +0000 UTC m=+5.910441217 container cleanup ffe9aa9c8813136bbd113eb0581bea8185c52e8fc471ccbcc56927ac37f59c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-21T12:45:26.666 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:26 smithi072 bash[60531]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2 2024-05-21T12:45:27.032 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:26 smithi072 podman[60553]: 2024-05-21 12:45:26.697279846 +0000 UTC m=+0.616276819 container remove ffe9aa9c8813136bbd113eb0581bea8185c52e8fc471ccbcc56927ac37f59c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-21T12:45:27.347 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:27 smithi072 podman[60644]: 2024-05-21 12:45:27.032533819 +0000 UTC m=+0.108159357 container create 51dea185bf56a5991f42e0a1fe78bc15ed46a81bb6f1d03f32ceac66026ee79c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-21T12:45:27.347 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:27 smithi072 podman[60644]: 2024-05-21 12:45:26.941629174 +0000 UTC m=+0.017254719 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:45:27.347 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:27 smithi072 podman[60644]: 2024-05-21 12:45:27.255050114 +0000 UTC m=+0.330675665 container init 51dea185bf56a5991f42e0a1fe78bc15ed46a81bb6f1d03f32ceac66026ee79c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b) 2024-05-21T12:45:27.347 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:27 smithi072 podman[60644]: 2024-05-21 12:45:27.258597009 +0000 UTC m=+0.334222541 container start 51dea185bf56a5991f42e0a1fe78bc15ed46a81bb6f1d03f32ceac66026ee79c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-21T12:45:27.347 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:27 smithi072 podman[60644]: 2024-05-21 12:45:27.346956849 +0000 UTC m=+0.422582388 container attach 51dea185bf56a5991f42e0a1fe78bc15ed46a81bb6f1d03f32ceac66026ee79c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T12:45:27.666 INFO:journalctl@ceph.osd.2.smithi072.stdout:May 21 12:45:27 smithi072 podman[60644]: 2024-05-21 12:45:27.553312294 +0000 UTC m=+0.628937835 container died 51dea185bf56a5991f42e0a1fe78bc15ed46a81bb6f1d03f32ceac66026ee79c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=ceph-nvmeof-mon-b0c764b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T12:45:27.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~15s 2024-05-21T12:45:27.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2024-05-21T12:45:27.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~25s 2024-05-21T12:45:27.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~25s 2024-05-21T12:45:27.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~20s 2024-05-21T12:45:27.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~20s 2024-05-21T12:45:28.176 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-21T12:45:28.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:28.215 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-21T12:45:28.215 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-21T12:45:28.215 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.3 2024-05-21T12:45:28.416 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:28 smithi072 systemd[1]: Stopping Ceph osd.3 for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:28.916 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:28 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3[42945]: 2024-05-21T12:45:28.576+0000 7f89166e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:28.916 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:28 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3[42945]: 2024-05-21T12:45:28.576+0000 7f89166e2700 -1 osd.3 163 *** Got signal Terminated *** 2024-05-21T12:45:28.916 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:28 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3[42945]: 2024-05-21T12:45:28.576+0000 7f89166e2700 -1 osd.3 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T12:45:31.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:31 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:31.155+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:32.385 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:32 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:32.108+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:32.793 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~20s 2024-05-21T12:45:32.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~10s 2024-05-21T12:45:32.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-21T12:45:32.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~30s 2024-05-21T12:45:32.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~30s 2024-05-21T12:45:32.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~25s 2024-05-21T12:45:32.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~25s 2024-05-21T12:45:33.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:33 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:33.138+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:34.266 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:34 smithi072 podman[60751]: 2024-05-21 12:45:34.011204177 +0000 UTC m=+5.543268196 container died 8144547dcfbde3370d869df75e81318fe3152ba75456d48d03c6b4884f8fb056 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-21T12:45:34.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:34.161+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:34.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:34.355+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:34.642 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:34 smithi072 podman[60751]: 2024-05-21 12:45:34.35550804 +0000 UTC m=+5.887572052 container cleanup 8144547dcfbde3370d869df75e81318fe3152ba75456d48d03c6b4884f8fb056 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T12:45:34.642 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:34 smithi072 bash[60751]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3 2024-05-21T12:45:34.916 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:34 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:34.739+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:09.390781+0000 front 2024-05-21T12:45:09.390877+0000 (oldest deadline 2024-05-21T12:45:34.090894+0000) 2024-05-21T12:45:34.916 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:34 smithi072 podman[60786]: 2024-05-21 12:45:34.641809593 +0000 UTC m=+0.635072863 container remove 8144547dcfbde3370d869df75e81318fe3152ba75456d48d03c6b4884f8fb056 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T12:45:35.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:34.652+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:35.283 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:35 smithi072 podman[60879]: 2024-05-21 12:45:34.907061667 +0000 UTC m=+0.023858191 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:45:35.283 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:35 smithi072 podman[60879]: 2024-05-21 12:45:35.010243349 +0000 UTC m=+0.127039854 container create 0ccda87ea3a4477b300b79b23d37fa97ae9db2395fa44e939b2d02e37084d074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-21T12:45:35.283 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:35 smithi072 podman[60879]: 2024-05-21 12:45:35.207413451 +0000 UTC m=+0.324209968 container init 0ccda87ea3a4477b300b79b23d37fa97ae9db2395fa44e939b2d02e37084d074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b) 2024-05-21T12:45:35.283 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:35 smithi072 podman[60879]: 2024-05-21 12:45:35.210558091 +0000 UTC m=+0.327354604 container start 0ccda87ea3a4477b300b79b23d37fa97ae9db2395fa44e939b2d02e37084d074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=ceph-nvmeof-mon-b0c764b) 2024-05-21T12:45:35.283 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:35 smithi072 podman[60879]: 2024-05-21 12:45:35.282870591 +0000 UTC m=+0.399667095 container attach 0ccda87ea3a4477b300b79b23d37fa97ae9db2395fa44e939b2d02e37084d074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-21T12:45:35.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:35.203+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:35.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:35.315+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:35.666 INFO:journalctl@ceph.osd.3.smithi072.stdout:May 21 12:45:35 smithi072 podman[60879]: 2024-05-21 12:45:35.50970374 +0000 UTC m=+0.626500259 container died 0ccda87ea3a4477b300b79b23d37fa97ae9db2395fa44e939b2d02e37084d074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=ceph-nvmeof-mon-b0c764b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T12:45:36.002 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:35 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:35.619+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:36.099 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:35 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:35.741+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:09.390781+0000 front 2024-05-21T12:45:09.390877+0000 (oldest deadline 2024-05-21T12:45:34.090894+0000) 2024-05-21T12:45:36.115 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-21T12:45:36.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:36.152 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-21T12:45:36.152 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-21T12:45:36.152 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.4 2024-05-21T12:45:36.416 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:36 smithi072 systemd[1]: Stopping Ceph osd.4 for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:36.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:36 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:36.316+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:36.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:36 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:36.153+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:36.778 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:36.520+0000 7fb39235c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:36.778 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:36.520+0000 7fb39235c700 -1 osd.4 163 *** Got signal Terminated *** 2024-05-21T12:45:36.778 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:36.520+0000 7fb39235c700 -1 osd.4 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T12:45:37.002 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:36 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:36.577+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:37.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:36 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:36.778+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:09.390781+0000 front 2024-05-21T12:45:09.390877+0000 (oldest deadline 2024-05-21T12:45:34.090894+0000) 2024-05-21T12:45:37.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:37 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:37.188+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:37.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:37 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:37.337+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:37.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~25s 2024-05-21T12:45:37.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~15s 2024-05-21T12:45:37.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2024-05-21T12:45:37.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-21T12:45:37.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~35s 2024-05-21T12:45:37.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~35s 2024-05-21T12:45:37.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~30s 2024-05-21T12:45:37.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~30s 2024-05-21T12:45:38.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:37 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:37.579+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:38.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:37 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:37.736+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:09.390781+0000 front 2024-05-21T12:45:09.390877+0000 (oldest deadline 2024-05-21T12:45:34.090894+0000) 2024-05-21T12:45:38.502 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:38.183+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:38.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:38.367+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:39.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:38 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:38.614+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:39.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:38 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:38.723+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:09.390781+0000 front 2024-05-21T12:45:09.390877+0000 (oldest deadline 2024-05-21T12:45:34.090894+0000) 2024-05-21T12:45:39.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:39 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:39.183+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:39.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:39 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:39.412+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:39.916 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:39 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:39.729+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:09.390781+0000 front 2024-05-21T12:45:09.390877+0000 (oldest deadline 2024-05-21T12:45:34.090894+0000) 2024-05-21T12:45:39.916 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:39 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:39.729+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:17.591469+0000 front 2024-05-21T12:45:17.591496+0000 (oldest deadline 2024-05-21T12:45:39.291516+0000) 2024-05-21T12:45:40.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:39 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:39.646+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:40.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:40.194+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:40.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:40.371+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:41.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:40.609+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:41.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:40 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:40.609+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:41.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:40 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:40.731+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:09.390781+0000 front 2024-05-21T12:45:09.390877+0000 (oldest deadline 2024-05-21T12:45:34.090894+0000) 2024-05-21T12:45:41.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:40 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4[48541]: 2024-05-21T12:45:40.731+0000 7fb38a948700 -1 osd.4 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:17.591469+0000 front 2024-05-21T12:45:17.591496+0000 (oldest deadline 2024-05-21T12:45:39.291516+0000) 2024-05-21T12:45:41.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:41.230+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:41.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:41.230+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:41.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:41.331+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:42.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:41.586+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:42.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:41.586+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:42.131 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:41 smithi072 podman[60983]: 2024-05-21 12:45:41.8661535 +0000 UTC m=+5.478978597 container died 0ec103327324b3cc2fa940326909204d73b7bbb76098d24b5fb1da2fc0bace4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-05-21T12:45:42.392 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:42 smithi072 podman[60983]: 2024-05-21 12:45:42.205439164 +0000 UTC m=+5.818264258 container cleanup 0ec103327324b3cc2fa940326909204d73b7bbb76098d24b5fb1da2fc0bace4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-21T12:45:42.392 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:42 smithi072 bash[60983]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4 2024-05-21T12:45:42.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:42.186+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:42.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:42.186+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:42.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:42.326+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:42.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:42.326+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:16.906141+0000 front 2024-05-21T12:45:16.906240+0000 (oldest deadline 2024-05-21T12:45:41.606191+0000) 2024-05-21T12:45:42.666 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:42 smithi072 podman[61009]: 2024-05-21 12:45:42.392593803 +0000 UTC m=+0.525797658 container remove 0ec103327324b3cc2fa940326909204d73b7bbb76098d24b5fb1da2fc0bace4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T12:45:42.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~30s 2024-05-21T12:45:42.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~20s 2024-05-21T12:45:42.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~10s 2024-05-21T12:45:42.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~5s 2024-05-21T12:45:42.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~40s 2024-05-21T12:45:42.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~40s 2024-05-21T12:45:42.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~35s 2024-05-21T12:45:42.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~35s 2024-05-21T12:45:43.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:42.567+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:43.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:42 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:42.567+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:43.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:42 smithi072 podman[61099]: 2024-05-21 12:45:42.827705025 +0000 UTC m=+0.109906434 container create 71ae1d86daad0c10504b2f7722f8ac41053601be97434b41a3a7b951af572476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-21T12:45:43.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:42 smithi072 podman[61099]: 2024-05-21 12:45:42.734783411 +0000 UTC m=+0.016984824 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:45:43.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:42 smithi072 podman[61099]: 2024-05-21 12:45:42.989764266 +0000 UTC m=+0.271965667 container init 71ae1d86daad0c10504b2f7722f8ac41053601be97434b41a3a7b951af572476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-21T12:45:43.166 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:42 smithi072 podman[61099]: 2024-05-21 12:45:42.993027726 +0000 UTC m=+0.275229142 container start 71ae1d86daad0c10504b2f7722f8ac41053601be97434b41a3a7b951af572476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T12:45:43.167 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:43 smithi072 podman[61099]: 2024-05-21 12:45:43.073560874 +0000 UTC m=+0.355762292 container attach 71ae1d86daad0c10504b2f7722f8ac41053601be97434b41a3a7b951af572476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-05-21T12:45:43.345 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:43.185+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:43.345 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:43.185+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:43.580 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:43 smithi072 podman[61099]: 2024-05-21 12:45:43.297145959 +0000 UTC m=+0.579347376 container died 71ae1d86daad0c10504b2f7722f8ac41053601be97434b41a3a7b951af572476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-21T12:45:43.654 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:43.556+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:43.654 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:43.556+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:43.654 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:43.345+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:43.655 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:43 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:43.345+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:16.906141+0000 front 2024-05-21T12:45:16.906240+0000 (oldest deadline 2024-05-21T12:45:41.606191+0000) 2024-05-21T12:45:43.833 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 21 12:45:43 smithi072 podman[61124]: 2024-05-21 12:45:43.814894202 +0000 UTC m=+0.512185679 container remove 71ae1d86daad0c10504b2f7722f8ac41053601be97434b41a3a7b951af572476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=ceph-nvmeof-mon-b0c764b, io.buildah.version=1.33.5) 2024-05-21T12:45:43.838 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-21T12:45:43.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:43.876 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-21T12:45:43.876 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-21T12:45:43.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.5 2024-05-21T12:45:44.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:44.136+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:44.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:44.136+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:44.253 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:43 smithi092 systemd[1]: Stopping Ceph osd.5 for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:44.253 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:44.160+0000 7f8ba1f85700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:44.253 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:44.160+0000 7f8ba1f85700 -1 osd.5 163 *** Got signal Terminated *** 2024-05-21T12:45:44.253 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:44.160+0000 7f8ba1f85700 -1 osd.5 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T12:45:44.753 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:44.537+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:44.753 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:44.537+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:44.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:44.359+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:44.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:44 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:44.359+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:16.906141+0000 front 2024-05-21T12:45:16.906240+0000 (oldest deadline 2024-05-21T12:45:41.606191+0000) 2024-05-21T12:45:45.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:45.356+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:45.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:45.356+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:16.906141+0000 front 2024-05-21T12:45:16.906240+0000 (oldest deadline 2024-05-21T12:45:41.606191+0000) 2024-05-21T12:45:45.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:45.148+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:45.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:45.148+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:46.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:45.554+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:46.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:45 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:45.554+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:46.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:46.134+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:46.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:46.134+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:46.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:46.134+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:46.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:46.337+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:46.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:46.337+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:16.906141+0000 front 2024-05-21T12:45:16.906240+0000 (oldest deadline 2024-05-21T12:45:41.606191+0000) 2024-05-21T12:45:46.503 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:46.337+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:23.306927+0000 front 2024-05-21T12:45:23.306942+0000 (oldest deadline 2024-05-21T12:45:46.206876+0000) 2024-05-21T12:45:47.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:46.525+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:47.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:46 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:46.525+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:47.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:47.109+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:47.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:47.109+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:47.364 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:47.109+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:47.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:47.362+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:47.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:47.362+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:16.906141+0000 front 2024-05-21T12:45:16.906240+0000 (oldest deadline 2024-05-21T12:45:41.606191+0000) 2024-05-21T12:45:47.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:47.362+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:23.306927+0000 front 2024-05-21T12:45:23.306942+0000 (oldest deadline 2024-05-21T12:45:46.206876+0000) 2024-05-21T12:45:47.753 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:47.544+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:47.753 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:47.544+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:47.753 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:47 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:47.544+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~35s 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~25s 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~15s 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~10s 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~45s 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~45s 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~40s 2024-05-21T12:45:47.799 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~40s 2024-05-21T12:45:48.359 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:48.092+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:48.359 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:48.092+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:48.359 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:48.092+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:48.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:48.357+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:08.105463+0000 front 2024-05-21T12:45:08.105452+0000 (oldest deadline 2024-05-21T12:45:34.005497+0000) 2024-05-21T12:45:48.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:48.357+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:16.906141+0000 front 2024-05-21T12:45:16.906240+0000 (oldest deadline 2024-05-21T12:45:41.606191+0000) 2024-05-21T12:45:48.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5[37871]: 2024-05-21T12:45:48.357+0000 7f8b9e579700 -1 osd.5 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:23.306927+0000 front 2024-05-21T12:45:23.306942+0000 (oldest deadline 2024-05-21T12:45:46.206876+0000) 2024-05-21T12:45:48.753 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:48.551+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:48.753 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:48.551+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:48.753 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:48 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:48.551+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:49.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:49.059+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:49.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:49.059+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:49.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:49.059+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:49.819 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:49 smithi092 podman[62100]: 2024-05-21 12:45:49.556649491 +0000 UTC m=+5.494228304 container died 53fc0870708f3e39191ed0f1043cfa767dee04aead8a62c0d2419a5cced7b6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=ceph-nvmeof-mon-b0c764b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-21T12:45:49.820 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:49.599+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:49.820 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:49.599+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:49.820 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:49 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:49.599+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:50.089 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:49 smithi092 podman[62100]: 2024-05-21 12:45:49.918378683 +0000 UTC m=+5.855957496 container cleanup 53fc0870708f3e39191ed0f1043cfa767dee04aead8a62c0d2419a5cced7b6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-21T12:45:50.090 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:49 smithi092 bash[62100]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5 2024-05-21T12:45:50.172 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:50 smithi092 podman[62126]: 2024-05-21 12:45:50.169999295 +0000 UTC m=+0.608072787 container remove 53fc0870708f3e39191ed0f1043cfa767dee04aead8a62c0d2419a5cced7b6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-21T12:45:50.172 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:50.088+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:50.173 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:50.088+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:50.173 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:50.088+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:50.753 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:50 smithi092 podman[62213]: 2024-05-21 12:45:50.495960159 +0000 UTC m=+0.106007522 container create bb5247264ce5f748af080e6511eeebc47addeb0c90aaa32f63086644121931de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-21T12:45:50.754 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:50 smithi092 podman[62213]: 2024-05-21 12:45:50.406899385 +0000 UTC m=+0.016946761 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:45:50.754 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:50 smithi092 podman[62213]: 2024-05-21 12:45:50.673734581 +0000 UTC m=+0.283781956 container init bb5247264ce5f748af080e6511eeebc47addeb0c90aaa32f63086644121931de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-05-21T12:45:50.754 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:50 smithi092 podman[62213]: 2024-05-21 12:45:50.677269874 +0000 UTC m=+0.287317242 container start bb5247264ce5f748af080e6511eeebc47addeb0c90aaa32f63086644121931de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-21T12:45:50.754 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:50.633+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:50.754 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:50.633+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:50.754 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:50 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:50.633+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:51.043 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:50 smithi092 podman[62213]: 2024-05-21 12:45:50.758870513 +0000 UTC m=+0.368917900 container attach bb5247264ce5f748af080e6511eeebc47addeb0c90aaa32f63086644121931de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=ceph-nvmeof-mon-b0c764b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-21T12:45:51.044 INFO:journalctl@ceph.osd.5.smithi092.stdout:May 21 12:45:50 smithi092 podman[62213]: 2024-05-21 12:45:50.98201585 +0000 UTC m=+0.592063220 container died bb5247264ce5f748af080e6511eeebc47addeb0c90aaa32f63086644121931de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T12:45:51.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:51.042+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:51.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:51.042+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:51.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:51.042+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:51.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-21T12:45:51.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:51.588 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-21T12:45:51.588 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-21T12:45:51.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.6 2024-05-21T12:45:51.815 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:51.604+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:51.815 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:51.604+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:51.816 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:51.604+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:51.816 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:51 smithi092 systemd[1]: Stopping Ceph osd.6 for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:45:52.070 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:51.964+0000 7f1114e11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:45:52.071 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:51.964+0000 7f1114e11700 -1 osd.6 163 *** Got signal Terminated *** 2024-05-21T12:45:52.071 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:51 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:51.964+0000 7f1114e11700 -1 osd.6 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T12:45:52.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:52.069+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:52.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:52.069+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:52.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:52.069+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:52.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~40s 2024-05-21T12:45:52.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~30s 2024-05-21T12:45:52.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~20s 2024-05-21T12:45:52.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~15s 2024-05-21T12:45:52.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~5s 2024-05-21T12:45:52.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-21T12:45:52.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~50s 2024-05-21T12:45:52.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~50s 2024-05-21T12:45:52.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~45s 2024-05-21T12:45:52.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~45s 2024-05-21T12:45:53.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:52.646+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:53.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:52.646+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:53.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:52 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:52.646+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:53.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:53.043+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:53.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:53.043+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:53.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:53.043+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:54.002 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:53.678+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:54.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:53.678+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:54.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:53 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:53.678+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:54.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:54.027+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:54.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:54.027+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:54.503 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:54.027+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:54.980 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:54.679+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:54.980 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:54.679+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:54.980 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:54.679+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:54.980 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:54.679+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:31.585895+0000 front 2024-05-21T12:45:31.585928+0000 (oldest deadline 2024-05-21T12:45:54.485868+0000) 2024-05-21T12:45:55.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:54.978+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:55.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:54.978+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:55.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:54.978+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:55.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:54 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:54.978+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:45:55.987 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:55.642+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:55.987 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:55.642+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:55.987 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:55.642+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:55.987 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:55.642+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:31.585895+0000 front 2024-05-21T12:45:31.585928+0000 (oldest deadline 2024-05-21T12:45:54.485868+0000) 2024-05-21T12:45:56.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:55.985+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:56.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:55.985+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:56.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:55.985+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:56.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:55 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:55.985+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:45:57.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:56.652+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:05.283551+0000 front 2024-05-21T12:45:05.283568+0000 (oldest deadline 2024-05-21T12:45:34.083929+0000) 2024-05-21T12:45:57.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:56.652+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.084278+0000 front 2024-05-21T12:45:14.084351+0000 (oldest deadline 2024-05-21T12:45:39.984226+0000) 2024-05-21T12:45:57.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:56.652+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:22.884922+0000 front 2024-05-21T12:45:22.885032+0000 (oldest deadline 2024-05-21T12:45:46.984909+0000) 2024-05-21T12:45:57.004 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:56 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6[43732]: 2024-05-21T12:45:56.652+0000 7f1111405700 -1 osd.6 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:31.585895+0000 front 2024-05-21T12:45:31.585928+0000 (oldest deadline 2024-05-21T12:45:54.485868+0000) 2024-05-21T12:45:57.429 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:57.016+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:57.430 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:57.016+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:57.430 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:57.016+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:57.430 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:57.016+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:45:57.698 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:57 smithi092 podman[62320]: 2024-05-21 12:45:57.429359838 +0000 UTC m=+5.581796630 container died a15b4b6a2f7af719293e99e948ab73375c1052ca14435a5f2d0ebb1fde62d062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~45s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~35s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~25s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~20s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~10s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~5s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~55s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~55s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~50s 2024-05-21T12:45:57.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~50s 2024-05-21T12:45:57.980 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:57 smithi092 podman[62320]: 2024-05-21 12:45:57.772565726 +0000 UTC m=+5.925002515 container cleanup a15b4b6a2f7af719293e99e948ab73375c1052ca14435a5f2d0ebb1fde62d062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T12:45:57.980 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:57 smithi092 bash[62320]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6 2024-05-21T12:45:58.253 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:57 smithi092 podman[62356]: 2024-05-21 12:45:57.99173684 +0000 UTC m=+0.557012881 container remove a15b4b6a2f7af719293e99e948ab73375c1052ca14435a5f2d0ebb1fde62d062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=ceph-nvmeof-mon-b0c764b) 2024-05-21T12:45:58.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:57.978+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:58.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:57.978+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:58.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:57.978+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:58.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:57 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:57.978+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:45:58.695 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:58 smithi092 podman[62444]: 2024-05-21 12:45:58.32772222 +0000 UTC m=+0.017357401 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:45:58.695 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:58 smithi092 podman[62444]: 2024-05-21 12:45:58.425778695 +0000 UTC m=+0.115413861 container create 9c37244a477e34fd78c5dcbdd52a176d34245985e1486acd25862d3ad9d482e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-21T12:45:58.695 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:58 smithi092 podman[62444]: 2024-05-21 12:45:58.69479284 +0000 UTC m=+0.384428023 container init 9c37244a477e34fd78c5dcbdd52a176d34245985e1486acd25862d3ad9d482e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-21T12:45:59.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:58 smithi092 podman[62444]: 2024-05-21 12:45:58.698178978 +0000 UTC m=+0.387814161 container start 9c37244a477e34fd78c5dcbdd52a176d34245985e1486acd25862d3ad9d482e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-21T12:45:59.003 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:58 smithi092 podman[62444]: 2024-05-21 12:45:58.769776695 +0000 UTC m=+0.459411890 container attach 9c37244a477e34fd78c5dcbdd52a176d34245985e1486acd25862d3ad9d482e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=ceph-nvmeof-mon-b0c764b) 2024-05-21T12:45:59.289 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:59 smithi092 podman[62444]: 2024-05-21 12:45:59.007640804 +0000 UTC m=+0.697275975 container died 9c37244a477e34fd78c5dcbdd52a176d34245985e1486acd25862d3ad9d482e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b) 2024-05-21T12:45:59.290 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:59.015+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:45:59.290 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:59.015+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:45:59.290 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:59.015+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:45:59.290 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:59.015+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:45:59.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-21T12:45:59.555 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:59 smithi092 podman[62470]: 2024-05-21 12:45:59.515675658 +0000 UTC m=+0.501822444 container remove 9c37244a477e34fd78c5dcbdd52a176d34245985e1486acd25862d3ad9d482e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-21T12:45:59.556 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:59 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.6.service: Deactivated successfully. 2024-05-21T12:45:59.556 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:59 smithi092 systemd[1]: Stopped Ceph osd.6 for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:45:59.556 INFO:journalctl@ceph.osd.6.smithi092.stdout:May 21 12:45:59 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.6.service: Consumed 1min 24.505s CPU time. 2024-05-21T12:45:59.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:45:59.570 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-21T12:45:59.571 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-21T12:45:59.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.7 2024-05-21T12:45:59.894 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:59 smithi092 systemd[1]: Stopping Ceph osd.7 for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:46:00.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:59.892+0000 7fb054744700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T12:46:00.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:59.892+0000 7fb054744700 -1 osd.7 163 *** Got signal Terminated *** 2024-05-21T12:46:00.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:45:59 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:45:59.892+0000 7fb054744700 -1 osd.7 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T12:46:00.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:00.037+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:46:00.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:00.037+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:46:00.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:00.037+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:46:00.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:00 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:00.037+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:46:01.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:01.000+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:46:01.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:01.000+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:46:01.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:01.000+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:46:01.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:01.000+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:46:02.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:01.955+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:46:02.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:01.955+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:46:02.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:01.955+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:46:02.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:01 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:01.955+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:46:02.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~50s 2024-05-21T12:46:02.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~40s 2024-05-21T12:46:02.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~30s 2024-05-21T12:46:02.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~25s 2024-05-21T12:46:02.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~15s 2024-05-21T12:46:02.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~10s 2024-05-21T12:46:02.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-21T12:46:02.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~60s 2024-05-21T12:46:02.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~60s 2024-05-21T12:46:02.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~55s 2024-05-21T12:46:02.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~55s 2024-05-21T12:46:03.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:02 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:02.930+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:46:03.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:02 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:02.930+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:46:03.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:02 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:02.930+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:46:03.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:02 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:02.930+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:46:04.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:03.946+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-05-21T12:45:07.884006+0000 front 2024-05-21T12:45:07.883935+0000 (oldest deadline 2024-05-21T12:45:30.783912+0000) 2024-05-21T12:46:04.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:03.946+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.45:6814 osd.1 since back 2024-05-21T12:45:14.884532+0000 front 2024-05-21T12:45:14.884501+0000 (oldest deadline 2024-05-21T12:45:40.784490+0000) 2024-05-21T12:46:04.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:03.946+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6804 osd.2 since back 2024-05-21T12:45:20.784795+0000 front 2024-05-21T12:45:20.784904+0000 (oldest deadline 2024-05-21T12:45:46.084786+0000) 2024-05-21T12:46:04.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:03 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7[49421]: 2024-05-21T12:46:03.946+0000 7fb04cd30700 -1 osd.7 163 heartbeat_check: no reply from 172.21.15.72:6812 osd.3 since back 2024-05-21T12:45:28.985299+0000 front 2024-05-21T12:45:28.985355+0000 (oldest deadline 2024-05-21T12:45:54.285327+0000) 2024-05-21T12:46:05.619 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:05 smithi092 podman[62554]: 2024-05-21 12:46:05.347456824 +0000 UTC m=+5.543916586 container died 30c13c67ca7b238e8071abade7294a1a1f132fd8bccd803d45dc0962f2d42bbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=ceph-nvmeof-mon-b0c764b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-21T12:46:05.937 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:05 smithi092 podman[62554]: 2024-05-21 12:46:05.693941628 +0000 UTC m=+5.890401389 container cleanup 30c13c67ca7b238e8071abade7294a1a1f132fd8bccd803d45dc0962f2d42bbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T12:46:05.937 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:05 smithi092 bash[62554]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7 2024-05-21T12:46:06.253 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:05 smithi092 podman[62580]: 2024-05-21 12:46:05.936526931 +0000 UTC m=+0.588531056 container remove 30c13c67ca7b238e8071abade7294a1a1f132fd8bccd803d45dc0962f2d42bbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=ceph-nvmeof-mon-b0c764b) 2024-05-21T12:46:06.591 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:06 smithi092 podman[62666]: 2024-05-21 12:46:06.182056701 +0000 UTC m=+0.017146859 image pull 1238c5f3fda9f25d1f4139e52ea412d8b792fe6f24ae443b8b9419e67b347815 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93 2024-05-21T12:46:06.591 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:06 smithi092 podman[62666]: 2024-05-21 12:46:06.290437247 +0000 UTC m=+0.125527404 container create 5470595e73b4249010d4239d678383161f9c8ece417e7d4b4f797ca207f10668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7-deactivate, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-21T12:46:06.591 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:06 smithi092 podman[62666]: 2024-05-21 12:46:06.507626355 +0000 UTC m=+0.342716528 container init 5470595e73b4249010d4239d678383161f9c8ece417e7d4b4f797ca207f10668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_CLEAN=True) 2024-05-21T12:46:06.591 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:06 smithi092 podman[62666]: 2024-05-21 12:46:06.511113427 +0000 UTC m=+0.346203591 container start 5470595e73b4249010d4239d678383161f9c8ece417e7d4b4f797ca207f10668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-21T12:46:06.591 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:06 smithi092 podman[62666]: 2024-05-21 12:46:06.590419655 +0000 UTC m=+0.425509829 container attach 5470595e73b4249010d4239d678383161f9c8ece417e7d4b4f797ca207f10668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=ceph-nvmeof-mon-b0c764b, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-21T12:46:07.003 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:06 smithi092 podman[62666]: 2024-05-21 12:46:06.818500212 +0000 UTC m=+0.653590377 container died 5470595e73b4249010d4239d678383161f9c8ece417e7d4b4f797ca207f10668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7-deactivate, RELEASE=ceph-nvmeof-mon-b0c764b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-21T12:46:07.362 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 21 12:46:07 smithi092 podman[62691]: 2024-05-21 12:46:07.343250017 +0000 UTC m=+0.518855118 container remove 5470595e73b4249010d4239d678383161f9c8ece417e7d4b4f797ca207f10668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:91ce839ff63281b6f657e35417aa29a2a46b64eaaa802c308dd1ee5dfeb05e93, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=ceph-nvmeof-mon-b0c764b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T12:46:07.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-21T12:46:07.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T12:46:07.404 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-21T12:46:07.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T12:46:07.591 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:46:07.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~55s 2024-05-21T12:46:07.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~45s 2024-05-21T12:46:07.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~35s 2024-05-21T12:46:07.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~30s 2024-05-21T12:46:07.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~20s 2024-05-21T12:46:07.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~15s 2024-05-21T12:46:07.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~5s 2024-05-21T12:46:07.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-21T12:46:07.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~65s 2024-05-21T12:46:07.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~65s 2024-05-21T12:46:07.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~60s 2024-05-21T12:46:07.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~60s 2024-05-21T12:46:08.813 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:08 smithi045 systemd[1]: Stopping Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:46:09.067 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:08 smithi045 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a[77017]: [21-May-2024 12:46:08] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.a/nvmeof-log.gz 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~60s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~50s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~40s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~35s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~25s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~20s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~10s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~5s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~70s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~70s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~65s 2024-05-21T12:46:12.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~65s 2024-05-21T12:46:17.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~65s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~55s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~45s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~40s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~30s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~25s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~15s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~10s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~75s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~75s 2024-05-21T12:46:17.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~70s 2024-05-21T12:46:17.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~70s 2024-05-21T12:46:19.169 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:18 smithi045 bash[80578]: time="2024-05-21T12:46:18Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a in 10 seconds, resorting to SIGKILL" 2024-05-21T12:46:19.169 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:18 smithi045 podman[80578]: 2024-05-21 12:46:18.845092492 +0000 UTC m=+10.130700211 container died 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, io.ceph.git.branch=tags/1.2.9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.git.branch=undefined, io.ceph.nvmeof=None, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.build.target-arch=, io.spdk.build.makeflags=, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., maintainer=Ceph Developers , io.spdk.build.pkgdep-args=--rbd, io.ceph.component=ceph-nvmeof, org.centos.version=9.0-21.el9, io.spdk.git.repo=https://github.com/ceph/spdk.git, version=1.2.9, com.redhat.component=ubi9-container, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, architecture=x86_64, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, vcs-type=git, summary=Ceph NVMe over Fabrics Gateway, release=, io.spdk.build.ceph-release=19.0.0, io.ceph.url=https://github.com/ceph/ceph-nvmeof, name=ceph-nvmeof, io.ceph.maintainer=Ceph Developers , vendor=Red Hat, Inc., io.spdk.version=24.01, io.buildah.version=1.29.0, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.openshift.wants=ceph,rbd, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., build-date=2024-05-19 17:57:19 UTC, distribution-scope=public, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., url=https://github.com/ceph/ceph-nvmeof) 2024-05-21T12:46:19.445 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:19 smithi045 podman[80578]: 2024-05-21 12:46:19.243708043 +0000 UTC m=+10.529315752 container cleanup 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, vcs-type=git, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, build-date=2024-05-19 17:57:19 UTC, io.ceph.git.branch=tags/1.2.9, name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Ceph Developers , io.spdk.build.pkgdep-args=--rbd, io.openshift.wants=ceph,rbd, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.branch=undefined, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.spdk.version=24.01, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.target-arch=, version=1.2.9, distribution-scope=public, url=https://github.com/ceph/ceph-nvmeof, io.ceph.version=1.2.9, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.summary=Ceph NVMe over Fabrics Gateway, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., summary=Ceph NVMe over Fabrics Gateway, io.ceph.component=ceph-nvmeof, io.spdk.build.ceph-release=19.0.0, com.redhat.component=ubi9-container, vendor=Red Hat, Inc., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.buildah.version=1.29.0, io.spdk.git.repo=https://github.com/ceph/spdk.git, org.centos.version=9.0-21.el9, io.ceph.maintainer=Ceph Developers , io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.makeflags=, architecture=x86_64, io.ceph.nvmeof=None, release=, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF).) 2024-05-21T12:46:19.445 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:19 smithi045 bash[80578]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a 2024-05-21T12:46:19.734 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:19 smithi045 podman[80609]: 2024-05-21 12:46:19.444736511 +0000 UTC m=+0.594843450 container remove 40977b6eabfc5d3f2646f069c8e89e0544ed9651d1bfe207eca44b0ccedd863f (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-a, distribution-scope=public, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.git.repo=https://github.com/ceph/spdk.git, release=, io.spdk.build.ceph-release=19.0.0, io.ceph.maintainer=Ceph Developers , io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, build-date=2024-05-19 17:57:19 UTC, io.buildah.version=1.29.0, io.spdk.version=24.01, io.spdk.build.makeflags=, name=ceph-nvmeof, io.spdk.git.branch=undefined, io.spdk.build.pkgdep-args=--rbd, version=1.2.9, org.centos.version=9.0-21.el9, io.spdk.build.target-arch=, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.openshift.wants=ceph,rbd, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.k8s.display-name=Ceph NVMe over Fabrics Gateway, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, vendor=Red Hat, Inc., org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.url=https://github.com/ceph/ceph-nvmeof, url=https://github.com/ceph/ceph-nvmeof, com.redhat.component=ubi9-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.component=ceph-nvmeof, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, architecture=x86_64, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.version=1.2.9, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, maintainer=Ceph Developers , io.ceph.git.branch=tags/1.2.9, vcs-type=git, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.nvmeof=None, summary=Ceph NVMe over Fabrics Gateway, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF).) 2024-05-21T12:46:19.734 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:19 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:46:19.734 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:19 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Failed with result 'exit-code'. 2024-05-21T12:46:19.734 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:19 smithi045 systemd[1]: Stopped Ceph nvmeof.nvmeof.a for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:46:19.734 INFO:journalctl@ceph.nvmeof.nvmeof.a.smithi045.stdout:May 21 12:46:19 smithi045 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.a.service: Consumed 6min 31.376s CPU time. 2024-05-21T12:46:20.316 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T12:46:20.504 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:46:21.666 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:21 smithi072 systemd[1]: Stopping Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:46:22.166 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:21 smithi072 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b[56971]: [21-May-2024 12:46:21] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.b/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.b/nvmeof-log.gz 2024-05-21T12:46:22.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~70s 2024-05-21T12:46:22.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~60s 2024-05-21T12:46:22.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~50s 2024-05-21T12:46:22.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~45s 2024-05-21T12:46:22.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~35s 2024-05-21T12:46:22.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~30s 2024-05-21T12:46:22.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~20s 2024-05-21T12:46:22.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~15s 2024-05-21T12:46:22.813 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~80s 2024-05-21T12:46:22.813 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~80s 2024-05-21T12:46:22.813 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~75s 2024-05-21T12:46:22.813 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~75s 2024-05-21T12:46:27.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~75s 2024-05-21T12:46:27.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~65s 2024-05-21T12:46:27.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~55s 2024-05-21T12:46:27.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~50s 2024-05-21T12:46:27.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~40s 2024-05-21T12:46:27.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~35s 2024-05-21T12:46:27.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~25s 2024-05-21T12:46:27.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~20s 2024-05-21T12:46:27.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~85s 2024-05-21T12:46:27.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~85s 2024-05-21T12:46:27.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~80s 2024-05-21T12:46:27.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~80s 2024-05-21T12:46:31.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:31 smithi072 bash[61476]: time="2024-05-21T12:46:31Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b in 10 seconds, resorting to SIGKILL" 2024-05-21T12:46:31.916 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:31 smithi072 podman[61476]: 2024-05-21 12:46:31.708605815 +0000 UTC m=+10.115995362 container died 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, name=ceph-nvmeof, io.ceph.component=ceph-nvmeof, io.spdk.git.branch=undefined, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.buildah.version=1.29.0, vendor=Red Hat, Inc., org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, build-date=2024-05-19 17:57:19 UTC, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=, com.redhat.component=ubi9-container, io.openshift.wants=ceph,rbd, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.makeflags=, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, distribution-scope=public, io.ceph.version=1.2.9, summary=Ceph NVMe over Fabrics Gateway, io.spdk.build.pkgdep-args=--rbd, io.spdk.version=24.01, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.ceph-release=19.0.0, io.ceph.git.branch=tags/1.2.9, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, architecture=x86_64, vcs-type=git, maintainer=Ceph Developers , io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, org.centos.version=9.0-21.el9, version=1.2.9, io.ceph.nvmeof=None, url=https://github.com/ceph/ceph-nvmeof, io.spdk.build.target-arch=, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.ceph.maintainer=Ceph Developers , description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF).) 2024-05-21T12:46:32.216 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:32 smithi072 podman[61476]: 2024-05-21 12:46:32.028637633 +0000 UTC m=+10.436027167 container cleanup 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.ceph.git.branch=tags/1.2.9, io.spdk.build.makeflags=, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.version=1.2.9, io.ceph.component=ceph-nvmeof, io.openshift.wants=ceph,rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, summary=Ceph NVMe over Fabrics Gateway, io.ceph.summary=Ceph NVMe over Fabrics Gateway, version=1.2.9, distribution-scope=public, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.spdk.build.pkgdep-args=--rbd, io.ceph.maintainer=Ceph Developers , vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.buildah.version=1.29.0, url=https://github.com/ceph/ceph-nvmeof, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.repo=https://github.com/ceph/spdk.git, io.spdk.version=24.01, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, name=ceph-nvmeof, com.redhat.component=ubi9-container, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, architecture=x86_64, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.build.target-arch=, io.ceph.url=https://github.com/ceph/ceph-nvmeof, vcs-type=git, io.spdk.build.ceph-release=19.0.0, vendor=Red Hat, Inc., build-date=2024-05-19 17:57:19 UTC, release=, io.ceph.nvmeof=None, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, maintainer=Ceph Developers , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, org.centos.version=9.0-21.el9, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.git.branch=undefined) 2024-05-21T12:46:32.216 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:32 smithi072 bash[61476]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b 2024-05-21T12:46:32.468 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:32 smithi072 podman[61506]: 2024-05-21 12:46:32.215742967 +0000 UTC m=+0.507450568 container remove 8659e489daf55074daf00a4786424f74ff6b18997be3c085328c437bfe9e2096 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-b, io.ceph.nvmeof=None, io.ceph.version=1.2.9, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.pkgdep-args=--rbd, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.openshift.wants=ceph,rbd, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.ceph-release=19.0.0, release=, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.git.repo=https://github.com/ceph/spdk.git, maintainer=Ceph Developers , io.ceph.component=ceph-nvmeof, com.redhat.component=ubi9-container, io.ceph.maintainer=Ceph Developers , io.spdk.git.branch=undefined, vendor=Red Hat, Inc., distribution-scope=public, summary=Ceph NVMe over Fabrics Gateway, name=ceph-nvmeof, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.spdk.build.makeflags=, url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.branch=tags/1.2.9, io.ceph.summary=Ceph NVMe over Fabrics Gateway, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., vcs-type=git, version=1.2.9, io.spdk.version=24.01, architecture=x86_64, io.buildah.version=1.29.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., org.centos.version=9.0-21.el9, io.spdk.build.target-arch=, build-date=2024-05-19 17:57:19 UTC, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug) 2024-05-21T12:46:32.468 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:32 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Main process exited, code=exited, status=137/n/a 2024-05-21T12:46:32.469 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:32 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Failed with result 'exit-code'. 2024-05-21T12:46:32.469 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:32 smithi072 systemd[1]: Stopped Ceph nvmeof.nvmeof.b for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:46:32.469 INFO:journalctl@ceph.nvmeof.nvmeof.b.smithi072.stdout:May 21 12:46:32 smithi072 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.b.service: Consumed 10min 30.490s CPU time. 2024-05-21T12:46:32.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~80s 2024-05-21T12:46:32.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~70s 2024-05-21T12:46:32.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~60s 2024-05-21T12:46:32.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~55s 2024-05-21T12:46:32.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~45s 2024-05-21T12:46:32.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2024-05-21T12:46:32.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~30s 2024-05-21T12:46:32.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~25s 2024-05-21T12:46:32.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~90s 2024-05-21T12:46:32.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~90s 2024-05-21T12:46:32.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~85s 2024-05-21T12:46:32.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~85s 2024-05-21T12:46:33.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T12:46:33.247 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:46:34.487 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:34 smithi092 systemd[1]: Stopping Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968... 2024-05-21T12:46:34.753 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:34 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:34] INFO utils.py:392: Will compress log file /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log to /var/log/ceph/nvmeof-client.nvmeof.nvmeof.c/nvmeof-log.gz 2024-05-21T12:46:37.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~85s 2024-05-21T12:46:37.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~75s 2024-05-21T12:46:37.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2024-05-21T12:46:37.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~60s 2024-05-21T12:46:37.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~50s 2024-05-21T12:46:37.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~45s 2024-05-21T12:46:37.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~35s 2024-05-21T12:46:37.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~30s 2024-05-21T12:46:37.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~95s 2024-05-21T12:46:37.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~95s 2024-05-21T12:46:37.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~90s 2024-05-21T12:46:37.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~90s 2024-05-21T12:46:41.754 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: terminate called after throwing an instance of 'std::runtime_error' 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: what(): Lost connection to the monitor (beacon timeout). 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: *** Caught signal (Aborted) ** 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: in thread 7fbfb1ed6640 thread_name:safe_timer 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: ceph version 19.0.0-3665-gb0c764b6 (b0c764b6e22fbb51f59e469d9dd99895c152f73e) squid (dev) 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbfbb2d26f0] 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 2: /lib64/libc.so.6(+0x8b94c) [0x7fbfbb31f94c] 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 3: raise() 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 4: abort() 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 5: /lib64/libstdc++.so.6(+0xa1b21) [0x7fbfbb636b21] 2024-05-21T12:46:41.755 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 6: /lib64/libstdc++.so.6(+0xad52c) [0x7fbfbb64252c] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 7: /lib64/libstdc++.so.6(+0xad597) [0x7fbfbb642597] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 8: /lib64/libstdc++.so.6(+0xad7f9) [0x7fbfbb6427f9] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 9: /usr/bin/ceph-nvmeof-monitor-client(+0x9e62a) [0x557e33d9a62a] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 10: (NVMeofGwMonitorClient::tick()+0xea) [0x557e33e2ca7a] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 11: /usr/bin/ceph-nvmeof-monitor-client(+0x11decd) [0x557e33e19ecd] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 12: (CommonSafeTimer::timer_thread()+0x133) [0x7fbfbbf9cc03] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 13: /usr/lib64/ceph/libceph-common.so.2(+0x27a1b1) [0x7fbfbbfa11b1] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 14: /lib64/libc.so.6(+0x89c02) [0x7fbfbb31dc02] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: 15: /lib64/libc.so.6(+0x10ec40) [0x7fbfbb3a2c40] 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] INFO utils.py:258: Initialize gateway log level to "INFO" 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] ERROR server.py:42: GatewayServer: SIGCHLD received signum=17 2024-05-21T12:46:41.756 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] ERROR server.py:46: PID of terminated child process is 19 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] ERROR server.py:110: GatewayServer exception occurred: 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: Traceback (most recent call last): 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: File "/src/control/__main__.py", line 45, in 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: gateway.keep_alive() 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: File "/src/control/server.py", line 491, in keep_alive 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: time.sleep(self.spdk_ping_interval_in_seconds) 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: File "/src/control/server.py", line 55, in sigchld_handler 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: raise SystemExit(f"Gateway subprocess terminated {pid=} {exit_code=}") 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: SystemExit: Gateway subprocess terminated pid=19 exit_code=-6 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] INFO server.py:411: Aborting (client.nvmeof.nvmeof.c) pid 19... 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] INFO server.py:411: Aborting (client.nvmeof.nvmeof.c) pid 70... 2024-05-21T12:46:41.757 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] INFO server.py:124: Stopping the server... 2024-05-21T12:46:41.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] INFO server.py:443: Terminating discovery service... 2024-05-21T12:46:41.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] INFO server.py:450: Discovery service terminated 2024-05-21T12:46:41.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: [21-May-2024 12:46:41] INFO server.py:131: Exiting the gateway process. 2024-05-21T12:46:41.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c[60232]: Gateway subprocess terminated pid=19 exit_code=-6 2024-05-21T12:46:41.758 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:41 smithi092 podman[63024]: 2024-05-21 12:46:41.502565199 +0000 UTC m=+7.126867705 container died 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, version=1.2.9, io.spdk.build.ceph-release=19.0.0, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.spdk.build.target-arch=, com.redhat.component=ubi9-container, io.openshift.wants=ceph,rbd, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, summary=Ceph NVMe over Fabrics Gateway, architecture=x86_64, release=, vcs-type=git, org.centos.version=9.0-21.el9, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, name=ceph-nvmeof, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.version=1.2.9, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.ceph.git.branch=tags/1.2.9, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, build-date=2024-05-19 17:57:19 UTC, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.build.pkgdep-args=--rbd, io.spdk.git.branch=undefined, io.buildah.version=1.29.0, io.spdk.build.makeflags=, io.ceph.component=ceph-nvmeof, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., distribution-scope=public, io.ceph.maintainer=Ceph Developers , maintainer=Ceph Developers , io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, url=https://github.com/ceph/ceph-nvmeof, io.spdk.version=24.01, io.spdk.git.repo=https://github.com/ceph/spdk.git, vendor=Red Hat, Inc., io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.nvmeof=None) 2024-05-21T12:46:42.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:42 smithi092 podman[63024]: 2024-05-21 12:46:42.049246161 +0000 UTC m=+7.673548663 container cleanup 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, io.ceph.component=ceph-nvmeof, io.ceph.version=1.2.9, io.spdk.build.target-arch=, com.redhat.component=ubi9-container, io.spdk.build.ceph-release=19.0.0, io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.git.branch=tags/1.2.9, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, vcs-type=git, org.centos.version=9.0-21.el9, url=https://github.com/ceph/ceph-nvmeof, io.buildah.version=1.29.0, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, architecture=x86_64, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.openshift.wants=ceph,rbd, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., build-date=2024-05-19 17:57:19 UTC, distribution-scope=public, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.ceph.maintainer=Ceph Developers , io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.git.repo=https://github.com/ceph/spdk.git, vendor=Red Hat, Inc., io.ceph.url=https://github.com/ceph/ceph-nvmeof, version=1.2.9, io.spdk.build.makeflags=, name=ceph-nvmeof, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.spdk.git.branch=undefined, release=, io.ceph.nvmeof=None, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, summary=Ceph NVMe over Fabrics Gateway, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.spdk.build.pkgdep-args=--rbd, io.spdk.version=24.01, maintainer=Ceph Developers ) 2024-05-21T12:46:42.253 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:42 smithi092 bash[63024]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c 2024-05-21T12:46:42.614 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:42 smithi092 podman[63045]: 2024-05-21 12:46:42.350510996 +0000 UTC m=+0.846517674 container remove 062cde6ce92f8f08183335174f5b16c50fab4837429250690518087100db22f4 (image=quay.io/ceph/nvmeof:1.2, name=ceph-c7e3752a-176b-11ef-bc9a-c7b262605968-nvmeof-nvmeof-c, com.redhat.component=ubi9-container, io.ceph.nvmeof=None, io.spdk.git.commit=dade5b4c96a647e63ef96d9a5e05f3b64ea0598a, version=1.2.9, io.ceph.git.branch=tags/1.2.9, io.ceph.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., distribution-scope=public, name=ceph-nvmeof, io.spdk.version=24.01, io.ceph.summary=Ceph NVMe over Fabrics Gateway, io.openshift.expose-services=4420/tcp:nvme,5500/tcp:grpc,8009/tcp:nvme-disc,10008/tcp:prom, io.openshift.tags=ceph,nvme-of,nvme-of gateway,rbd,block storage, io.ceph.maintainer=Ceph Developers , io.ceph.version=1.2.9, io.spdk.build.ceph-release=19.0.0, io.spdk.git.branch=undefined, io.ceph.url=https://github.com/ceph/ceph-nvmeof, io.k8s.description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., io.ceph.git.commit=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, maintainer=Ceph Developers , release=, architecture=x86_64, io.spdk.build.makeflags=, io.spdk.build.pkgdep-args=--rbd, url=https://github.com/ceph/ceph-nvmeof, summary=Ceph NVMe over Fabrics Gateway, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Service to provide block storage on top of Ceph for platforms (e.g.: VMWare) without native Ceph support (RBD), replacing existing approaches (iSCSI) with a newer and more versatile standard (NVMe-oF)., org.centos.version=9.0-21.el9, vcs-ref=06ef3a2cf9a75d1eefaaaed7bf8cf627b4aa2cec, io.buildah.version=1.29.0, build-date=2024-05-19 17:57:19 UTC, org.centos.url=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/Packages/, io.ceph.component=ceph-nvmeof, io.openshift.wants=ceph,rbd, io.spdk.build.target-arch=, io.k8s.display-name=Ceph NVMe over Fabrics Gateway, io.spdk.git.repo=https://github.com/ceph/spdk.git, io.ceph.git.repo=https://github.com/ceph/ceph-nvmeof, io.spdk.build.configure-args=--with-rbd --disable-tests --disable-unit-tests --disable-examples --enable-debug, vendor=Red Hat, Inc.) 2024-05-21T12:46:42.614 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:42 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Main process exited, code=exited, status=1/FAILURE 2024-05-21T12:46:42.614 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:42 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Failed with result 'exit-code'. 2024-05-21T12:46:42.614 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:42 smithi092 systemd[1]: Stopped Ceph nvmeof.nvmeof.c for c7e3752a-176b-11ef-bc9a-c7b262605968. 2024-05-21T12:46:42.614 INFO:journalctl@ceph.nvmeof.nvmeof.c.smithi092.stdout:May 21 12:46:42 smithi092 systemd[1]: ceph-c7e3752a-176b-11ef-bc9a-c7b262605968@nvmeof.nvmeof.c.service: Consumed 3min 2.914s CPU time. 2024-05-21T12:46:42.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~90s 2024-05-21T12:46:42.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~80s 2024-05-21T12:46:42.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~70s 2024-05-21T12:46:42.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~65s 2024-05-21T12:46:42.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~55s 2024-05-21T12:46:42.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~50s 2024-05-21T12:46:42.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~40s 2024-05-21T12:46:42.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~35s 2024-05-21T12:46:42.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~100s 2024-05-21T12:46:42.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~100s 2024-05-21T12:46:42.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~95s 2024-05-21T12:46:42.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~95s 2024-05-21T12:46:43.110 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T12:46:43.299 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:46:43.914 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:46:43.951 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:46:43.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:46:44.013 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T12:46:44.040 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-21T12:46:44.042 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi045/crash 2024-05-21T12:46:44.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T12:46:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T12:46:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-05-21T12:46:44.084 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi072/crash 2024-05-21T12:46:44.085 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T12:46:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:tar: /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T12:46:44.129 INFO:teuthology.orchestra.run.smithi072.stderr:tar: Error is not recoverable: exiting now 2024-05-21T12:46:44.131 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi092/crash 2024-05-21T12:46:44.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T12:46:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T12:46:44.181 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-05-21T12:46:44.182 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi137/crash 2024-05-21T12:46:44.183 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T12:46:44.221 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T12:46:44.221 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-05-21T12:46:44.222 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-21T12:46:44.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_FAILED_DAEMON | egrep -v 'is in error state' | egrep -v 'failed cephadm daemon' | head -n 1 2024-05-21T12:46:44.265 INFO:tasks.cephadm:Compressing logs... 2024-05-21T12:46:44.265 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T12:46:44.308 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T12:46:44.310 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T12:46:44.313 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T12:46:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T12:46:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:: No such file or directory 2024-05-21T12:46:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T12:46:44.333 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-21T12:46:44.333 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-21T12:46:44.333 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-21T12:46:44.334 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-21T12:46:44.334 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.0.log: /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T12:46:44.336 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T12:46:44.336 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T12:46:44.336 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T12:46:44.336 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T12:46:44.336 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T12:46:44.338 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T12:46:44.338 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-05-21T12:46:44.338 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T12:46:44.339 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 86.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T12:46:44.339 INFO:teuthology.orchestra.run.smithi137.stderr: 90.0% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T12:46:44.339 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T12:46:44.339 INFO:teuthology.orchestra.run.smithi092.stderr:: No such file or directory 2024-05-21T12:46:44.339 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T12:46:44.339 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-21T12:46:44.340 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-21T12:46:44.340 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-21T12:46:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/cephadm.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T12:46:44.342 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-05-21T12:46:44.342 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m0.012s 2024-05-21T12:46:44.343 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m0.010s 2024-05-21T12:46:44.343 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m0.013s 2024-05-21T12:46:44.345 INFO:teuthology.orchestra.run.smithi072.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log: No such file or directory 2024-05-21T12:46:44.345 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-05-21T12:46:44.345 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-21T12:46:44.345 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T12:46:44.345 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.3.log: /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr: 81.6% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.b.log: 91.3% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/cephadm.log: 87.9% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T12:46:44.346 INFO:teuthology.orchestra.run.smithi072.stderr: 89.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T12:46:44.347 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T12:46:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-21T12:46:44.348 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.5.log: /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T12:46:44.348 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T12:46:44.348 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T12:46:44.348 INFO:teuthology.orchestra.run.smithi092.stderr: 81.5% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T12:46:44.348 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log: 87.9% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T12:46:44.348 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T12:46:44.348 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/cephadm.log: 90.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T12:46:44.348 INFO:teuthology.orchestra.run.smithi092.stderr: 91.2% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T12:46:44.349 INFO:teuthology.orchestra.run.smithi072.stderr: 91.2% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T12:46:44.623 INFO:teuthology.orchestra.run.smithi072.stderr: 92.7% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-21T12:46:44.631 INFO:teuthology.orchestra.run.smithi092.stderr: 92.7% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-21T12:46:44.718 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-21T12:46:45.315 INFO:teuthology.orchestra.run.smithi045.stderr: 90.5% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-21T12:46:47.822 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~95s 2024-05-21T12:46:47.822 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~85s 2024-05-21T12:46:47.822 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~75s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~70s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~60s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~55s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~45s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~40s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~105s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~105s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~100s 2024-05-21T12:46:47.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~100s 2024-05-21T12:46:49.191 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-21T12:46:49.823 INFO:teuthology.orchestra.run.smithi072.stderr: 93.6% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-21T12:46:51.269 INFO:teuthology.orchestra.run.smithi092.stderr: 93.5% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~100s 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~90s 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~80s 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~75s 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~65s 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~60s 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~50s 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~45s 2024-05-21T12:46:52.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~110s 2024-05-21T12:46:52.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~110s 2024-05-21T12:46:52.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~105s 2024-05-21T12:46:52.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~105s 2024-05-21T12:46:53.343 INFO:teuthology.orchestra.run.smithi072.stderr: 93.6% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-21T12:46:53.670 INFO:teuthology.orchestra.run.smithi092.stderr: 93.5% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-21T12:46:55.858 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-21T12:46:55.860 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-05-21T12:46:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m11.535s 2024-05-21T12:46:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m25.226s 2024-05-21T12:46:55.860 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m0.906s 2024-05-21T12:46:56.565 INFO:teuthology.orchestra.run.smithi045.stderr: 93.7% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-21T12:46:56.567 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-21T12:46:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m12.246s 2024-05-21T12:46:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m17.910s 2024-05-21T12:46:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.561s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~105s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~95s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~85s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~80s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~70s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~65s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~55s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~50s 2024-05-21T12:46:57.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~115s 2024-05-21T12:46:57.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~115s 2024-05-21T12:46:57.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~110s 2024-05-21T12:46:57.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~110s 2024-05-21T12:46:59.898 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/c7e3752a-176b-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-21T12:46:59.900 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-05-21T12:46:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m15.574s 2024-05-21T12:46:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m30.987s 2024-05-21T12:46:59.900 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m1.118s 2024-05-21T12:46:59.900 INFO:tasks.cephadm:Archiving logs... 2024-05-21T12:46:59.900 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi045/log 2024-05-21T12:46:59.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T12:47:02.507 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi072/log 2024-05-21T12:47:02.508 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T12:47:02.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~110s 2024-05-21T12:47:02.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~100s 2024-05-21T12:47:02.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~90s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~85s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~75s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~70s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~60s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~55s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~120s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~120s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~115s 2024-05-21T12:47:02.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~115s 2024-05-21T12:47:06.399 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi092/log 2024-05-21T12:47:06.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T12:47:07.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~115s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~105s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~95s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~90s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~80s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~75s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~65s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~60s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~125s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~125s 2024-05-21T12:47:07.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~120s 2024-05-21T12:47:07.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~120s 2024-05-21T12:47:11.273 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi137/log 2024-05-21T12:47:11.274 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T12:47:11.306 INFO:tasks.cephadm:Removing cluster... 2024-05-21T12:47:11.307 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --force 2024-05-21T12:47:11.510 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:47:11.801 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --force 2024-05-21T12:47:11.992 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:47:12.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --force 2024-05-21T12:47:12.466 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:47:12.760 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c7e3752a-176b-11ef-bc9a-c7b262605968 --force 2024-05-21T12:47:12.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~120s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~110s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~100s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~95s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~85s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~80s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~70s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~65s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~130s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~130s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~125s 2024-05-21T12:47:12.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~125s 2024-05-21T12:47:12.945 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: c7e3752a-176b-11ef-bc9a-c7b262605968 2024-05-21T12:47:13.344 INFO:tasks.cephadm:Removing cephadm ... 2024-05-21T12:47:13.345 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T12:47:13.365 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T12:47:13.383 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T12:47:13.400 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T12:47:13.416 INFO:tasks.cephadm:Teardown complete 2024-05-21T12:47:13.416 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-21T12:47:13.428 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-21T12:47:13.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T12:47:13.431 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T12:47:13.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T12:47:13.442 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T12:47:13.572 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-21T12:47:13.581 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-21T12:47:13.582 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T12:47:13.585 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T12:47:13.587 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T12:47:13.589 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T12:47:13.600 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-05-21T12:47:13.601 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-05-21T12:47:13.602 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-21T12:47:13.603 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-05-21T12:47:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T12:47:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-05-21T12:47:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +3158us[+3206us] +/- 47ms 2024-05-21T12:47:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -2607us[-2607us] +/- 77ms 2024-05-21T12:47:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -2664us[-2664us] +/- 77ms 2024-05-21T12:47:13.605 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:47:13.607 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T12:47:13.607 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-05-21T12:47:13.607 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +3197us[+3201us] +/- 47ms 2024-05-21T12:47:13.607 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -2550us[-2550us] +/- 77ms 2024-05-21T12:47:13.607 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -2611us[-2611us] +/- 77ms 2024-05-21T12:47:13.607 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +3204us[+3276us] +/- 47ms 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -2543us[-2472us] +/- 77ms 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -2609us[-2538us] +/- 77ms 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +3166us[+3181us] +/- 47ms 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -2575us[-2575us] +/- 77ms 2024-05-21T12:47:13.609 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -2648us[-2648us] +/- 77ms 2024-05-21T12:47:13.610 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T12:47:13.610 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-21T12:47:13.628 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-21T12:47:13.629 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-21T12:47:13.638 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T12:47:13.728 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T12:47:13.798 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T12:47:13.868 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T12:47:13.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T12:47:13.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:47:13.986 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T12:47:14.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:47:14.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T12:47:14.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:47:14.046 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T12:47:14.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:47:14.078 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-21T12:47:14.089 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-21T12:47:14.099 INFO:teuthology.task.internal:Duration was 2382.863880 seconds 2024-05-21T12:47:14.100 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-21T12:47:14.110 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-21T12:47:14.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T12:47:14.113 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T12:47:14.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T12:47:14.118 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T12:47:14.152 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T12:47:14.153 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T12:47:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T12:47:14.156 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T12:47:14.600 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-21T12:47:14.601 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-05-21T12:47:14.601 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T12:47:14.663 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2024-05-21T12:47:14.663 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T12:47:14.693 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-05-21T12:47:14.693 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T12:47:14.727 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-05-21T12:47:14.727 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T12:47:14.751 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-21T12:47:14.751 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T12:47:14.754 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T12:47:14.756 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T12:47:14.769 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T12:47:14.818 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-21T12:47:14.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T12:47:14.821 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T12:47:14.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T12:47:14.838 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T12:47:15.164 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-21T12:47:15.175 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-21T12:47:15.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T12:47:15.200 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T12:47:15.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T12:47:15.251 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T12:47:15.277 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-21T12:47:15.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T12:47:15.290 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T12:47:15.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T12:47:15.294 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T12:47:15.312 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-05-21T12:47:15.313 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2024-05-21T12:47:15.315 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-05-21T12:47:15.339 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-05-21T12:47:15.359 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T12:47:15.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:47:15.404 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T12:47:15.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:47:15.418 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T12:47:15.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:47:15.433 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T12:47:15.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T12:47:15.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-21T12:47:15.459 INFO:teuthology.task.internal:Transferring archived files... 2024-05-21T12:47:15.460 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi045 2024-05-21T12:47:15.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T12:47:15.502 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi072 2024-05-21T12:47:15.503 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T12:47:15.538 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi092 2024-05-21T12:47:15.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T12:47:15.573 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vallariag-2024-05-21_11:53:37-rbd:nvmeof-main-distro-default-smithi/7718477/remote/smithi137 2024-05-21T12:47:15.574 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T12:47:15.610 INFO:teuthology.task.internal:Removing archive directory... 2024-05-21T12:47:15.610 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T12:47:15.613 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T12:47:15.615 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T12:47:15.617 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T12:47:15.660 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-21T12:47:15.673 INFO:teuthology.task.internal:Not uploading archives. 2024-05-21T12:47:15.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-21T12:47:15.707 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-21T12:47:15.708 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T12:47:15.710 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T12:47:15.712 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T12:47:15.714 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T12:47:15.723 INFO:teuthology.orchestra.run.smithi045.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 12:47 /home/ubuntu/cephtest 2024-05-21T12:47:15.725 INFO:teuthology.orchestra.run.smithi072.stdout: 265567 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 12:47 /home/ubuntu/cephtest 2024-05-21T12:47:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 12:47 /home/ubuntu/cephtest 2024-05-21T12:47:15.729 INFO:teuthology.orchestra.run.smithi137.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 12:47 /home/ubuntu/cephtest 2024-05-21T12:47:15.731 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-21T12:47:15.742 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-21T12:47:15.767 INFO:teuthology.run:Summary data: description: rbd:nvmeof/thrash/{base/install centos_latest clusters/3-gateways-1-initiator conf/{disable-pool-app mon} gateway-initiator-setup/3-subsys-60-namespace thrashers/nvmeof_thrash workloads/fio} duration: 2382.8638803958893 flavor: default owner: scheduled_vallariag@teuthology success: true 2024-05-21T12:47:15.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T12:47:15.914 INFO:teuthology.run:pass